我正在尝试使用scrapy刮除登录页后面的表。登录页面是http://subscribers.footballguys.com/amember/login.php,而我正在尝试的网页是https://subscribers.footballguys.com/myfbg/myweeklycheatsheet.php。你知道吗
我试着去学习教程from scrapy's documentation和here,但是我没有得到任何回应(甚至连hello world都没有)。下面是我的代码。我也可以提供任何其他需要的信息。提前谢谢!你知道吗
import scrapy
class FbgQbSpider(scrapy.Spider):
name = 'fbg_qb'
allowed_domains = ['www.footballguys.com/']
start_urls = ['http://subscribers.footballguys.com/amember/login.php']
def parse(self, response):
return scrapy.FormRequest.from_response(
response,
formdata={'amember_login': 'example@gmail.com', 'amember_pass': 'examplepassword'},
callback=self.after_login
)
def after_login(self, response):
#check login success before going on
View(response)
if "authentication failed" in response.body:
self.logger.error("Login failed")
return
fetch("https://subscribers.footballguys.com/myfbg/myweeklycheatsheet.php")
players = response.css("span::text").extract()
for item in zip(players):
scraped_info = {
'player' : item[0]
}
yield scraped_info
print("hello world")
hello world
由于缩进问题而未打印。你知道吗相关问题 更多 >
编程相关推荐