擅长:python、mysql、java
<p>你可以建立自己的运营商“mysql_to_s3”,并将其作为插件添加到Airflow中。在</p>
<p>有一个操作员可以将数据从Mysql归档到gcs:</p>
<p><a href="https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/mysql_to_gcs.py" rel="nofollow noreferrer">mysql_to_gcs.py</a></p>
<p>您可以让所有的代码在<code>def _upload_to_gcs</code>上稍作修改,而使用s3}钩子:<a href="https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/S3_hook.py" rel="nofollow noreferrer">s3_hook.py</a>。在</p>
<p>有关自定义插件的文档:</p>
<p><a href="http://michal.karzynski.pl/blog/2017/03/19/developing-workflows-with-apache-airflow/" rel="nofollow noreferrer">Airflow plugins: Blog article</a></p>
<p><a href="https://airflow.incubator.apache.org/plugins.html" rel="nofollow noreferrer">Airflow plugins: Official documentation</a></p>