Pyspark和python2.7对我来说很好。 我安装了python3.5.1(从源代码安装) 我在终端中运行pyspark时遇到这个错误
Python 3.5.1 (default, Apr 25 2016, 12:41:28)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "/home/himaprasoon/apps/spark-1.6.0-bin-hadoop2.6/python/pyspark/shell.py", line 30, in <module>
import pyspark
File "/home/himaprasoon/apps/spark-1.6.0-bin-hadoop2.6/python/pyspark/__init__.py", line 41, in <module>
from pyspark.context import SparkContext
File "/home/himaprasoon/apps/spark-1.6.0-bin-hadoop2.6/python/pyspark/context.py", line 28, in <module>
from pyspark import accumulators
File "/home/himaprasoon/apps/spark-1.6.0-bin-hadoop2.6/python/pyspark/accumulators.py", line 98, in <module>
from pyspark.serializers import read_int, PickleSerializer
File "/home/himaprasoon/apps/spark-1.6.0-bin-hadoop2.6/python/pyspark/serializers.py", line 58, in <module>
import zlib
ImportError: No module named 'zlib'
我试过python3.4.3,也很好用
您是否检查过zlib是否确实存在于python安装中?它应该是默认的,但奇怪的事情发生了。在
您是否在.bashrc文件中提供了系统python3.5.1到“PYSPARK_PYTHON”的确切路径?在
这是我的PySpark提示符显示的内容。ApacheSparkVerison是2.1.1
注:我使用Anaconda3(python3.6.1)作为我的日常PySpark代码,我的PySpark_驱动程序设置为“jupyter”
上面的例子是我的默认系统python3.6
试试看
conda install -c conda-forge pyspark
如果您的问题仍然存在,您可能需要更改您的~/.basrc相关问题 更多 >
编程相关推荐