在Win 7 (64位)上安装Scrapy

7 投票
8 回答
7439 浏览
提问于 2025-04-17 15:12

我正在尝试为 Python 2.6 安装 scrapy,但似乎不太顺利。这里是我安装的包:

G:\Python26\Scripts>pip freeze
Scrapy==0.16.4
Twisted==12.3.0
libxml2-python==2.7.7
lxml==2.3.6
pyopenssl==0.13
w3lib==1.2
zope.interface==3.8.0

我还安装了 iconv 和 zlib。下面是我用 pip 安装 scrapy 时的日志。我不知道接下来该怎么做,是不是缺少了什么?需要一些指导,谢谢。

我的系统是 Windows 7 64位,已经安装了 Visual C++。

C:\Users\d>pip install scrapy
Downloading/unpacking scrapy
  Downloading Scrapy-0.16.4.tar.gz (679kB): 679kB downloaded
  Running setup.py egg_info for package scrapy
    warning: no files found matching 'license.txt' under directory 'scrapy'
    no previously-included directories found matching 'docs\build'
Requirement already satisfied (use --upgrade to upgrade): Twisted>=8.0 in g:\pyt
hon26\lib\site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.2 in g:\pytho
n26\lib\site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in g:\python26\li
b\site-packages\lxml-2.3.6-py2.6-win32.egg (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in g:\python
26\lib\site-packages\pyopenssl-0.13-py2.6-win32.egg (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0
in g:\python26\lib\site-packages\zope.interface-3.8.0-py2.6-win32.egg (from Twis
ted>=8.0->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in g:\pytho
n26\lib\site-packages (from zope.interface>=3.6.0->Twisted>=8.0->scrapy)
Installing collected packages: scrapy
  Running setup.py install for scrapy
    warning: no files found matching 'license.txt' under directory 'scrapy'
    no previously-included directories found matching 'docs\build'
Successfully installed scrapy
Cleaning up...


C:\Users\d>scrapy
Traceback (most recent call last):
  File "G:\Python26\lib\runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "G:\Python26\lib\runpy.py", line 34, in _run_code
    exec code in run_globals
  File "G:\Python26\lib\site-packages\scrapy\cmdline.py", line 7, in <module>
    from scrapy.crawler import CrawlerProcess
  File "G:\Python26\lib\site-packages\scrapy\crawler.py", line 5, in <module>
    from scrapy.core.engine import ExecutionEngine
  File "G:\Python26\lib\site-packages\scrapy\core\engine.py", line 14, in <modul
e>
    from scrapy.core.downloader import Downloader
  File "G:\Python26\lib\site-packages\scrapy\core\downloader\__init__.py", line
16, in <module>
    from .middleware import DownloaderMiddlewareManager
  File "G:\Python26\lib\site-packages\scrapy\core\downloader\middleware.py", lin
e 7, in <module>
    from scrapy.http import Request, Response
  File "G:\Python26\lib\site-packages\scrapy\http\__init__.py", line 11, in <mod
ule>
    from scrapy.http.request.form import FormRequest
  File "G:\Python26\lib\site-packages\scrapy\http\request\form.py", line 9, in <
module>
    import lxml.html
  File "G:\Python26\lib\site-packages\lxml-2.3.6-py2.6-win32.egg\lxml\html\__ini
t__.py", line 12, in <module>
    from lxml import etree
ImportError: DLL load failed: %1 is not a valid Win32 application

8 个回答

1

在我的情况下,我升级了Python,这导致了这个问题。
简单地运行一下 scrapy 升级命令就解决了这个问题。

pip install --upgrade Scrapy

另外,还要安装正确版本的 pywin: http://doc.scrapy.org/en/latest/intro/install.html#windows

1

可以看看这个关于在Windows 7上安装scrapy的指南。里面提到需要安装一个叫做egenix-pyopenssl的包,另外安装pywin32也可能会有帮助。

4

你需要按照这里的说明进行操作(上面也有列出)。

问题中提到的具体DLL错误,至少在我的情况下,是因为我安装了32位的OpenSSL,而其他的都是64位的。在我安装了正确的OpenSSL之后(顺便说一下,我还不得不卸载上面链接中的所有包,然后重新开始——安装顺序对依赖的msi安装程序很重要),现在在python27、windows7、64位上,scrapy运行得很好。

撰写回答