在python3中自动化云Firestore导出

Question

我想用一个基于flask的microservice为我的Cloud Firestore数据库建立一个自动备份服务，我需要使用以下命令：

gcloud beta firestore export gs://[BUCKET_NAME]

这是我想通过我的appengine微服务运行的命令

@app.route('/backup', methods=["GET", "POST"])
def backup():

    subprocess.call('gcloud beta firestore export gs://bucket-name --async', shell=True)

    return f"Backup process started successfully, you can close this window. {datetime.now(timezone.utc)}"

但是看起来什么都没有发生，我假设这是因为我的appengine实例没有CloudSDK。你知道吗

这是我可以在云函数中做的吗？你知道吗

Answer 1

不能在沙盒环境（appengine、functions）中执行系统调用。而且你不知道平台上安装了什么，这很危险/不一致。你知道吗

您可以尝试使用cloud run或app engine flex。但这不是一个真正的最佳实践。最好的方法是使用Python库以编程方式执行相同的操作。在任何情况下，底层结果都是相同的：一个API调用。你知道吗

Answer 2

下面是一个示例应用程序，您可以使用googleappengine Cron服务调用它。它基于node.js example in the docs：

附录yaml

runtime: python37

handlers:
- url: /.*
  script: auto

如果已经部署了默认服务，请添加target: cloud-firestore-admin以创建新服务。你知道吗

要求.txt

Flask
google-api-python-client

google-api-python-client简化了对CloudFireStoreRESTAPI的访问。你知道吗

主.py

import datetime
import os
from googleapiclient.discovery import build

from flask import Flask, request

app = Flask(__name__) 

@app.route('/cloud-firestore-export')
def export():
    # Deny if not from the GAE Cron Service
    assert request.headers['X-Appengine-Cron']
    # Deny if outputUriPrefix not set correctly
    outputUriPrefix = request.args.get('outputUriPrefix')
    assert outputUriPrefix and outputUriPrefix.startswith('gs://')
    # Use a timestamp in export file name
    timestamp = datetime.datetime.now().strftime('%Y%m%d-%H%M%S')
    if '/' not in outputUriPrefix[5:]:
      # Add a trailing slash if missing
      outputUriPrefix += '/' + timestamp
    else:
      outputUriPrefix += timestamp
    if 'collections' in request.args:
      collections = request.args.get('collections').split(",")
    else:
      collections = None

    body = {
        'collectionIds': collections,
        'outputUriPrefix': outputUriPrefix,
    }
    # Build REST API request for 
    # https://cloud.google.com/firestore/docs/reference/rest/v1/projects.databases/exportDocuments
    project_id = os.environ.get('GOOGLE_CLOUD_PROJECT')
    database_name = 'projects/{}/databases/(default)'.format(project_id)
    service = build('firestore', 'v1')
    service.projects().databases().exportDocuments(name=database_name, body=body).execute()
    return 'Operation started' 


if __name__ == '__main__':
    # This is used when running locally only. When deploying to Google App
    # Engine, a webserver process such as Gunicorn will serve the app. This
    # can be configured by adding an `entrypoint` to app.yaml.
    # Flask's development server will automatically serve static files in
    # the "static" directory. See:
    # http://flask.pocoo.org/docs/1.0/quickstart/#static-files. Once deployed,
    # App Engine itself will serve those files as configured in app.yaml.
    app.run(host='127.0.0.1', port=8080, debug=True)

克朗.亚马尔

cron:
- description: "Daily Cloud Firestore Export"
  url: /cloud-firestore-export?outputUriPrefix=gs://BUCKET_NAME&collections=COLLECTIONS_LIST
  schedule: every 24 hours

如果部署到中的非默认服务附录yaml，也添加到这里：target: cloud-firestore-admin。你知道吗

应用引擎服务帐户的访问权限

一旦部署，应用程序将使用GAE服务帐户来授权导出请求。确保您的GAE服务帐户对Cloud Firestore和存储桶具有权限，请参阅：

https://cloud.google.com/firestore/docs/solutions/schedule-export#configure_access_permissions