如果我在一个方法中多次调用此代码,它将失败,但终端中不会显示错误。它只运行一次。同一只蜘蛛不可能再爬两次吗?
它在生产线上失败了反应器.运行()并且spider不会在第二次调用时运行,但日志中没有错误。在
def crawlSite(self):
self.mySpider = MySpider()
self.mySpider.setCrawlFolder(self.website)
settings = get_project_settings()
settings.set('DEPTH_LIMIT', self.depth)
crawler = Crawler(settings)
crawler.signals.connect(reactor.stop, signal=signals.spider_closed)
crawler.configure()
crawler.crawl(self.mySpider)
crawler.start()
log.start(logfile="results.log", loglevel=log.ERROR, crawler=crawler, logstdout=False) #log.DEBUG
reactor.run() # the script will block here until the spider_closed signal was sent
这是神秘的课程
^{pr2}$然后我有一个SetupClass,它在crawlarClass中调用crawlSite()
self.crawlerClass.crawlSite()
目前没有回答
相关问题 更多 >
编程相关推荐