DAG未加载到airflow Web UI中

2024-05-23 16:44:44 发布

您现在位置:Python中文网/ 问答频道 /正文

我在试着做气流测试。它引用一个创建dataproc集群的模板。当我尝试运行DAG时,只显示DAG_id,其他字段保持空白。有人能解释一下为什么会发生这种事吗?或者我错在哪里了? 以下是我的模板:

{
    "id": "newone",
    "placement": {
        "managedCluster": {
            "clusterName": "testinstant",
            "config": {
                "gceClusterConfig": {
                    "zoneUri": "https://www.googleapis.com/compute/v1/projects/abhilash-thomas-sandbox/zones/us-central1-a",
                    "serviceAccountScopes": [
                        "https://www.googleapis.com/auth/cloud-platform/"
                    ]
                },
                "masterConfig": {
                    "numInstances": 1,
                    "machineTypeUri": "https://www.googleapis.com/compute/v1/projects/abhilash-thomas-sandbox/zones/us-central1-a/machineTypes/n1-standard-4",
                    "diskConfig": {
                        "bootDiskSizeGb": 100,
                        "bootDiskType": "pd-standard"
                    }
                },
                "workerConfig": {
                    "numInstances": 2,
                    "machineTypeUri": "https://www.googleapis.com/compute/v1/projects/abhilash-thomas-sandbox/zones/us-central1-a/machineTypes/n1-standard-4",
                    "diskConfig": {
                        "bootDiskSizeGb": 100,
                        "bootDiskType": "pd-standard"
                    }
                },
                "softwareConfig": {
                    "properties": {
                        "dataproc:dataproc.allow.zero.workers": "true"
                    }
                },
                "initializationActions": [{
                    "executableFile": "gs://loadtests/cloud-sql-proxy.sh",
                    "executionTimeout": "600s"
                }]
            }
        }
    },
    "jobs": [{
        "hiveJob": {
            "queryFileUri": "gs://loadtests/testplugin/queryfile.sql"
        },
        "stepId": "job0"
    }]
}

这是我的DAG文件:

^{pr2}$

Tags: httpscomwwwthomasstandardprojectsusdag