如何在ai中建立多个Dag目录

2024-05-13 22:44:35 发布

您现在位置:Python中文网/ 问答频道 /正文

我为不同的python项目设置了不同的气流dag,即一个父dags文件夹/vol/dags,其中包含基于不同python项目的dag子文件夹:/vol/dags/project1/project1.py, /vol/dags/project2/project2.py,其中DAGS_FOLDER = /vol/dags

例如,project1.py从同一目录中的另一个python文件导入一个函数,即/vol/dags/project1/mycalculator.py。但是当我启动airflow Web服务器时,我得到一个ImportError

/vol/dags/project1/$ airflow webserver -p 8080

INFO - Filling up the DagBag from /vol/dags/
ERROR - Failed to import: /vol/dags/project1/project1.py
Traceback (most recent call last):
  File "/Users/xxx/anaconda/lib/python2.7/site-packages/airflow/models.py", line 247, in process_file
    m = imp.load_source(mod_name, filepath)
  File "/vol/dags/project1/project1.py", line 10, in <module>
    from mycalculator import *
ImportError: No module named mycalculator

我试图像这样将mycalculator.py导入project1.py

from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators import PythonOperator
from datetime import datetime, timedelta
from mycalculator import *

dag = DAG(
    dag_id='project1', default_args=args,
    schedule_interval="@once")

Tags: 项目frompyimport文件夹linefileairflow