我正试图抓取一个需要登录的网站。问题是登录表单的url类似于以下内容:
/customer/guest/index
问题是如何将登录信息发送到此post方法
附言:我正在尝试使用文章中提到的方法
更新:以下是我正在使用的代码
from bs4 import BeautifulSoup
import requests
import csv
# Start the session
session = requests.Session()
# Create the payload
payload = {'CustomerLogin[email]':'<USERNAME>',
'CustomerLogin[password]':'<PASSWORD>',
'csrf_token': '<TOKEN>'
}
# Post the payload to the site to log in
s = session.post(" https://avangemail.net/customer/guest/index", data=payload)
print('Logging in...')
# Navigate to the next page and scrape the data
s = session.get('https://avangemail.net/customer/lists/all-subscribers')
print('Getting information...')
print(s.text)
soup = BeautifulSoup(s.text, 'html.parser')
table = soup.find('table')
tbody = table.find('tbody')
headers = [td.text for td in thead.select("tr td td")]
filename = 'avang-export'
with open(r'.'+filename+'.csv', "w") as f:
wr = csv.writer(f)
wr.writerows([[td.text for td in row.find_all("td")] for row in tbody.select("tr")])
然后将其传递给post方法
相关问题 更多 >
编程相关推荐