<p>根据<a href="https://github.com/apache/airflow/blob/master/airflow/contrib/hooks/gcp_dataflow_hook.py#L270" rel="nofollow noreferrer">hook source</a>,<code>machineType</code>是模板作业唯一可接受的密钥。然后,您指定的变量将用于构建对REST API的请求,如下所示:</p>
<pre class="lang-py prettyprint-override"><code># RuntimeEnvironment
environment = {}
for key in ['maxWorkers', 'zone', 'serviceAccountEmail', 'tempLocation',
'bypassTempDirValidation', 'machineType', 'network', 'subnetwork']:
if key in variables:
environment.update({key: variables[key]})
# LaunchTemplateParameters
body = {"jobName": name,
"parameters": parameters,
"environment": environment}
# projects.locations.template.launch
service = self.get_conn()
request = service.projects().locations().templates().launch(
projectId=variables['project'],
location=variables['region'],
gcsPath=dataflow_template,
body=body
)
</code></pre>
<p><a href="https://cloud.google.com/dataflow/docs/reference/rest/v1b3/projects.locations.templates/launch" rel="nofollow noreferrer">^{<cd2>}</a>的文档指定请求主体应该是<a href="https://cloud.google.com/dataflow/docs/reference/rest/v1b3/LaunchTemplateParameters" rel="nofollow noreferrer">^{<cd3>}</a>的一个实例,它有另一个{a4}。从hook源代码来看,这似乎是准确的。在</p>
<p>您可以采取一些调试步骤:您可以记录/检查传出的REST调用,或者在Stackdriver日志记录中找到该调用(以及与作业创建请求相关的元数据)。在</p>
<p>注意:这只在<a href="https://github.com/apache/airflow/commit/3c5b73579a3e6c8a1e47c2fddf201b99d690bafe" rel="nofollow noreferrer">[AIRFLOW-1954]</a>之后才可用,这是AirflowV1.10.0版本的一部分。这意味着它只存在于特定的cloudcomposer版本中。在</p>