Scrapy帮助在登录到pag后进行刮擦

2024-04-27 01:11:32 发布

您现在位置:Python中文网/ 问答频道 /正文

我正在尝试使用scrapy刮除登录页后面的表。登录页面是http://subscribers.footballguys.com/amember/login.php,而我正在尝试的网页是https://subscribers.footballguys.com/myfbg/myweeklycheatsheet.php。你知道吗

我试着去学习教程from scrapy's documentationhere,但是我没有得到任何回应(甚至连hello world都没有)。下面是我的代码。我也可以提供任何其他需要的信息。提前谢谢!你知道吗

import scrapy


class FbgQbSpider(scrapy.Spider):
    name = 'fbg_qb'
    allowed_domains = ['www.footballguys.com/']
    start_urls = ['http://subscribers.footballguys.com/amember/login.php']

    def parse(self, response):
        return scrapy.FormRequest.from_response(
            response,
            formdata={'amember_login': 'example@gmail.com', 'amember_pass': 'examplepassword'},
            callback=self.after_login
            )

    def after_login(self, response):
        #check login success before going on
        View(response)
        if "authentication failed" in response.body:
            self.logger.error("Login failed")
            return
        fetch("https://subscribers.footballguys.com/myfbg/myweeklycheatsheet.php")
        players = response.css("span::text").extract()

        for item in zip(players):
            scraped_info = {
                'player' : item[0]
                }
            yield scraped_info
            print("hello world")

Tags: fromhttpsselfcomhttphelloresponselogin