我将scrapy与selenium一起使用,代码是:
import scrapy
from scrapy.selector import Selector
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from shutil import which
class GoldendataSpider(scrapy.Spider):
name = 'goldendata'
allowed_domains = ['www.golden.com']
#start_urls = ['https://golden.com/wiki/Cluster%3A_COVID-19-ZXJX9AR/']
def __init__(self):
chrome_options= Options()
chrome_options.add_argument("--headless")
chrome_path=which('chromedriver')
driver = webdriver.Chrome(executable_path=chrome_path, options=chrome_options)
driver.get("https://golden.com/wiki/Cluster%3A_COVID-19-ZXJX9AR/")
self.html = driver.page_source
driver.close()
但我得到了以下错误:
Unhandled error in Deferred:
2020-05-02 10:11:35 [twisted] CRITICAL: Unhandled error in Deferred:
2020-05-02 10:19:16 [twisted] CRITICAL:
Traceback (most recent call last):
File "F:\Git Contribution\Fightpandemic\Scrpers\golden\golden\spiders\goldendata.py", line 22, in
__init__
driver = webdriver.Chrome(executable_path=chrome_path, chrome_options=chrome_options)
needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'NoneType' is not iterable
我试着安装pywin32并更新py和selenium,但没有任何效果。。 所以,我如何解决这个错误呢?我尝试了我知道的大多数方法,也搜索了github问题和堆栈溢出,但没有找到任何结果。 提前谢谢
您使用
which('chromedriver')
来获取chromedriver的路径,但是which
似乎返回None
,因为chromedriver不在路径中您可以在脚本中为
chrome_path
使用相对路径。或者,将chromedriver.exe添加到路径相关问题 更多 >
编程相关推荐