为Scrapy安装包依赖项
在使用Scrapy时,用户需要安装很多软件包,我觉得我在安装pyOpenSSL时遇到了问题。
当我尝试创建一个教程中的Scrapy项目时,出现了以下输出:
Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 168, in <module>
execute()
File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 122, in execute
cmds = _get_commands_dict(settings, inproject)
File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 46, in _get_comma
nds_dict
cmds = _get_commands_from_module('scrapy.commands', inproject)
File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 29, in _get_comma
nds_from_module
for cmd in _iter_command_classes(module):
File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 20, in _iter_comm
and_classes
for module in walk_modules(module_name):
File "C:\Python27\lib\site-packages\scrapy\utils\misc.py", line 68, in walk_mo
dules
submod = import_module(fullpath)
File "C:\Python27\lib\importlib\__init__.py", line 37, in import_module
__import__(name)
File "C:\Python27\lib\site-packages\scrapy\commands\bench.py", line 3, in <mod
ule>
from scrapy.tests.mockserver import MockServer
File "C:\Python27\lib\site-packages\scrapy\tests\mockserver.py", line 6, in <m
odule>
from twisted.internet import reactor, defer, ssl
File "C:\Python27\lib\site-packages\twisted\internet\ssl.py", line 59, in <mod
ule>
from OpenSSL import SSL
File "build\bdist.win32\egg\OpenSSL\__init__.py", line 8, in <module>
File "build\bdist.win32\egg\OpenSSL\rand.py", line 11, in <module>
File "build\bdist.win32\egg\OpenSSL\_util.py", line 3, in <module>
ImportError: No module named cryptography.hazmat.bindings.openssl.binding
然后我在网上搜索了最后一个错误(没有名为cryptography.hazmat的模块),发现有几处提到pyOpenSSL。所以我决定运行 easy_install pyOpenSSL==0.14
来确保安装的是最新版本,但执行后却得到了这个输出:
c:\python27\include\pymath.h(22) : warning C4273: 'round' : inconsistent dll lin
kage
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\math.h(51
6) : see previous definition of 'round'
c:\users\bk\appdata\local\temp\easy_install-tztawu\cryptography-0.4\temp\easy_in
stall-svxsjy\cffi-0.8.2\c\misc_win32.h(225) : error C2632: 'char' followed by 'b
ool' is illegal
c:\users\bk\appdata\local\temp\easy_install-tztawu\cryptography-0.4\temp\easy_in
stall-svxsjy\cffi-0.8.2\c\misc_win32.h(225) : warning C4091: 'typedef ' : ignore
d on left of 'unsigned char' when no variable is declared
c/_cffi_backend.c(5295) : warning C4146: unary minus operator applied to unsigne
d type, result still unsigned
c/_cffi_backend.c(5296) : warning C4146: unary minus operator applied to unsigne
d type, result still unsigned
c/_cffi_backend.c(5297) : warning C4146: unary minus operator applied to unsigne
d type, result still unsigned
c/_cffi_backend.c(5298) : warning C4146: unary minus operator applied to unsigne
d type, result still unsigned
error: Setup script exited with error: command '"C:\Program Files (x86)\Microsof
t Visual Studio 12.0\VC\BIN\cl.exe"' failed with exit status 2
所以我有点迷茫,不知道该怎么做才能让Scrapy正常运行。
4 个回答
0
在你尝试安装 Scrapy
之前,应该先升级一下 pip
:
pip install --upgrade pip
pip install Scrapy
0
我也遇到了同样的问题,试着用第一个答案来解决,但没成功。最后,我卸载了pyOpenSSL,然后重新下载并安装了pyopenssl。这样问题就解决了。pyopenssl的链接是:https://launchpad.net/pyopenssl
1
我强烈推荐在使用Windows时选择conda
,而不是pip
。因为conda
会自动为你的系统下载合适的二进制文件,这样设置一个科学计算的Python环境(比如Scipy、Numpy、Pandas等)就变得非常简单。
所以,先了解一下Anaconda,然后安装Anaconda,接着执行:
conda create -n scrapyenv python=2 # creates a new py2 environment
activate scrapyenv # switch to the new environment
conda install scrapy # install scrapy
前两步只有在你想把环境隔离开来时才需要做。顺便说一下,如果你执行conda install anaconda
,会安装一大堆有用的包。
另外,如果conda
没有包含pyOpenSSL
,或者你不想安装anaconda
,请查看教程的第9点,链接在这里:如何在64位Windows 7上安装Scrapy。
21
我在Mac OS上也遇到了同样的错误。
我通过使用openssl 0.13这个版本,而不是最新的版本,解决了这个问题。
easy_install pyOpenSSL==0.13
或者
pip install pyOpenSSL==0.13