apacheairflowprovidersapachespark for Apache Airflow提供程序包
apache-airflow-providers-apache-spark的Python项目详细描述
apache气流提供商apache spark包
版本：1.0.0
目录
提供程序包
这是apache.sparkprovider的提供程序包。此提供程序包的所有类
在airflow.providers.apache.sparkpython包中。在
安装
注意！在
2020年11月，新版本的PIP（20.3）发布了一个新的2020解析器。这个解析器
尚不适用于Apache Airflow，可能会导致安装错误-取决于您的选择
额外的。为了安装气流，您需要将pip降级到20.2.4版本
pip upgrade --pip==20.2.4或者，如果您使用pip20.3，您需要添加选项
--use-deprecated legacy-resolver到pip安装命令。在
您可以将此软件包安装在现有的airflow 2.*上，通过
pip install apache-airflow-providers-apache-spark
PIP要求
| PIP package | Version required |
|---|---|
| pyspark |
提供程序类摘要
在Airflow 2.0中，apache.spark提供者的所有操作员、传输、挂钩、传感器、机密
在airflow.providers.apache.spark包中。您可以阅读有关使用的命名约定的更多信息
在Naming conventions for provider packages
操作员
移动运算符
^{tb2}$钩子
移动挂钩
^{3}$释放
版本1.0.0
| Commit | Committed | Subject |
|---|---|---|
| b40dffa08 | 2020-12-08 | Rename remaing modules to match AIP-21 (#12917) |
| 9b39f2478 | 2020-12-08 | Add support for dynamic connection form fields per provider (#12558) |
| bd90136aa | 2020-11-30 | Move operator guides to provider documentation packages (#12681) |
| c34ef853c | 2020-11-20 | Separate out documentation building per provider (#12444) |
| 008035450 | 2020-11-18 | Update provider READMEs for 1.0.0b2 batch release (#12449) |
| ae7cb4a1e | 2020-11-17 | Update wrong commit hash in backport provider changes (#12390) |
| 6889a333c | 2020-11-15 | Improvements for operators and hooks ref docs (#12366) |
| 7825e8f59 | 2020-11-13 | Docs installation improvements (#12304) |
| 85a18e13d | 2020-11-09 | Point at pypi project pages for cross-dependency of provider packages (#12212) |
| 59eb5de78 | 2020-11-09 | Update provider READMEs for up-coming 1.0.0beta1 releases (#12206) |
| b2a28d159 | 2020-11-09 | Moves provider packages scripts to dev (#12082) |
| 4e8f9cc8d | 2020-11-03 | Enable Black - Python Auto Formmatter (#9550) |
| 8c42cf1b0 | 2020-11-03 | Use PyUpgrade to use Python 3.6 features (#11447) |
| 5a439e84e | 2020-10-26 | Prepare providers release 0.0.2a1 (#11855) |
| 872b1566a | 2020-10-25 | Generated backport providers readmes/setup for 2020.10.29 (#11826) |
| 349b0811c | 2020-10-20 | Add D200 pydocstyle check (#11688) |
| 16e712971 | 2020-10-13 | Added support for provider packages for Airflow 2.0 (#11487) |
| d305876be | 2020-10-12 | Remove redundant None provided as default to dict.get() (#11448) |
| 0a0e1af80 | 2020-10-03 | Fix Broken Markdown links in Providers README TOC (#11249) |
| ca4238eb4 | 2020-10-02 | Fixed month in backport packages to October (#11242) |
| 5220e4c38 | 2020-10-02 | Prepare Backport release 2020.09.07 (#11238) |
| f3e87c503 | 2020-09-22 | Add D202 pydocstyle check (#11032) |
| fdd9b6f65 | 2020-08-25 | Enable Black on Providers Packages (#10543) |
| d76026545 | 2020-08-25 | PyDocStyle: No whitespaces allowed surrounding docstring text (#10533) |
| d1bce91bb | 2020-08-25 | PyDocStyle: Enable D403: Capitalized first word of docstring (#10530) |
| 3696c34c2 | 2020-08-24 | Fix typo in the word "release" (#10528) |
| ee7ca128a | 2020-08-22 | Fix broken Markdown refernces in Providers README (#10483) |
| 7c206a82a | 2020-08-22 | Replace assigment with Augmented assignment (#10468) |
| 3b3287d7a | 2020-08-05 | Enforce keyword only arguments on apache operators (#10170) |
| 7d24b088c | 2020-07-25 | Stop using start_date in default_args in example_dags (2) (#9985) |
| 33f0cd265 | 2020-07-22 | apply_default keeps the function signature for mypy (#9784) |
| 1427e4acb | 2020-07-22 | Update Spark submit operator for Spark 3 support (#8730) |
| 4d74ac211 | 2020-07-19 | Increase typing for Apache and http provider package (#9729) |
| 0873070e0 | 2020-07-11 | Mask other forms of password arguments in SparkSubmitOperator (#9615) |
| 13a827d80 | 2020-07-09 | Ensure Kerberos token is valid in SparkSubmitOperator before running ^{ |
| 067806d59 | 2020-06-29 | Add tests for spark_jdbc_script (#9491) |
| d0e7db402 | 2020-06-19 | Fixed release number for fresh release (#9408) |
| 12af6a080 | 2020-06-19 | Final cleanup for 2020.6.23rc1 release preparation (#9404) |
| c7e5bce57 | 2020-06-19 | Prepare backport release candidate for 2020.6.23rc1 (#9370) |
| 40bf8f28f | 2020-06-18 | Detect automatically the lack of reference to the guide in the operator descriptions (#9290) |
| f6bd817a3 | 2020-06-16 | Introduce 'transfers' packages (#9320) |
| 0b0e4f7a4 | 2020-05-26 | Preparing for RC3 relase of backports (#9026) |
| 00642a46d | 2020-05-26 | Fixed name of 20 remaining wrongly named operators. (#8994) |
| 375d1ca22 | 2020-05-19 | Release candidate 2 for backport packages 2020.05.20 (#8898) |
| 12c5e5d8a | 2020-05-17 | Prepare release candidate for backport packages (#8891) |
| f3521fb0e | 2020-05-16 | Regenerate readme files for backport package release (#8886) |
| 92585ca4c | 2020-05-15 | Added automated release notes generation for backport operators (#8807) |
| 7506c73f1 | 2020-05-10 | Add default ^{ |
| 487b5cc50 | 2020-05-06 | Add guide for Apache Spark operators (#8305) |
| 87969a350 | 2020-04-09 | [AIRFLOW-6515] Change Log Levels from Info/Warn to Error (#8170) |
| be1451b0e | 2020-04-02 | [AIRFLOW-7026] Improve SparkSqlHook's error message (#7749) |
| 4bde99f13 | 2020-03-23 | Make airflow/providers pylint compatible (#7802) |
| 7e6372a68 | 2020-03-23 | Add call to Super call in apache providers (#7820) |
| 2327aa5a2 | 2020-03-12 | [AIRFLOW-7025] Fix SparkSqlHook.run_query to handle its parameter properly (#7677) |
| 024b4bf96 | 2020-03-10 | [AIRFLOW-7024] Add the verbose parameter support to SparkSqlOperator (#7676) |
| b59042b5a | 2020-02-28 | [AIRFLOW-6949] Respect explicit ^{ |
| 97a429f9d | 2020-02-02 | [AIRFLOW-6714] Remove magic comments about UTF-8 (#7338) |
| 0481b9a95 | 2020-01-12 | [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package (#7142) |
- 项目
标签：
欢迎加入QQ群-->： 979659372
