The Python packaging for Spark is not intended to replace all of the other use cases. This Python packaged version of Spark is suitable for interacting with an existing cluster (be it Spark standalone, YARN, or Mesos) - but does not contain the tools required to setup your own standalone Spark cluster. You can download the full version of Spark from the Apache Spark downloads page.
NOTE: If you are using this with a Spark standalone cluster you must
ensure that the version (including minor version) matches or you may
experience odd errors
虽然我还没有测试过,但是从spark2.1开始,PyPi可以提供PySpark(用于通过
pip
安装),这正是针对您这样的情况。从docs:相关问题 更多 >
编程相关推荐