如何使用post方法的相对URL将登录负载发送到表单?

2024-04-27 00:29:46 发布

您现在位置:Python中文网/ 问答频道 /正文

我正试图抓取一个需要登录的网站。问题是登录表单的url类似于以下内容:

/customer/guest/index

问题是如何将登录信息发送到此post方法

附言:我正在尝试使用文章中提到的方法

更新:以下是我正在使用的代码

from bs4 import BeautifulSoup
import requests
import csv


# Start the session
session = requests.Session()

# Create the payload
payload = {'CustomerLogin[email]':'<USERNAME>',
          'CustomerLogin[password]':'<PASSWORD>',
          'csrf_token': '<TOKEN>'
         }

# Post the payload to the site to log in
s = session.post(" https://avangemail.net/customer/guest/index", data=payload)
print('Logging in...')

# Navigate to the next page and scrape the data
s = session.get('https://avangemail.net/customer/lists/all-subscribers')

print('Getting information...')
print(s.text)
soup = BeautifulSoup(s.text, 'html.parser')
table = soup.find('table')
tbody = table.find('tbody')

headers = [td.text for td in thead.select("tr td td")]

filename = 'avang-export'
with open(r'.'+filename+'.csv', "w") as f:
    wr = csv.writer(f)
    wr.writerows([[td.text for td in row.find_all("td")] for row in tbody.select("tr")])

Tags: csvthetotextinimportforsession