使用Python2.7登录后如何填写和提交表单?

2024-05-13 10:50:58 发布

您现在位置:Python中文网/ 问答频道 /正文

我对Python和HTML语言非常陌生,我请求您帮助我完成以下任务(我在Stackoverflow上的第一篇文章…)。 我必须从我公司的网络工具中搜集所有客户的评论(不幸的是无法从外部访问)。 我使用以下脚本成功地登录到web工具:

import requests, lxml.html

s = requests.session()
login = s.get('http://bp-webtools1.otto.boreus.de/tools/login#')
login_html = lxml.html.fromstring(login.text)
hidden_inputs = login_html.xpath(r'//form//input[@type="hidden"]')   
#print hidden_inputs
form = {x.attrib['name']: x.attrib['value'] for x in hidden_inputs}
#print form
form2 = {'username': 'myusername',
         'password': 'mypassword',
         'utf': '✓',
         'authenticity_token': form["authenticity_token"], 
         'mandant': 'bp',
         'commit': 'Anmelden'
}
print form2
response = s.post('http://bp-webtools1.otto.boreus.de/tools/sessions', data=form2)
print(response.url)
print(response.status_code)
print(response.ok)
r = s.get('http://bp-webtools1.otto.boreus.de/tools/it/reviews/v2/reviews?review_service_host=live') #this is the page where I find the form to fill and submit
print r.content

登录之后,我应该填写一张表格,以便选择我感兴趣的评论的日期,提交它,然后刮去结果。我尝试应用与上面相同的逻辑,但我无法使post方法继续工作。你知道吗

如果我在填写和提交表单时使用Chrome开发者工具,我会得到以下结果:

Request URL:http://bp-webtools1.otto.boreus.de/tools/it/reviews/v2/reviews/search_reviews?review_service_host=live
Request Method:POST
Status Code:200 OK
Remote Address:10.107.10.40:80
Referrer Policy:no-referrer-when-downgrade
Response Headers
Cache-Control:no-cache, no-store, max-age=0, must-revalidate
Connection:Keep-Alive
Content-Type:text/javascript;charset=utf-8
Date:Wed, 10 Jan 2018 10:59:53 GMT
Expires:Fri, 01 Jan 1990 00:00:00 GMT
Keep-Alive:timeout=5, max=100
Pragma:no-cache
Server:Apache
Set-Cookie:_shoptools_session=UjcvTFQxMDdON2s4bmprb0JBZEhNeGc5b1VoT09TcUg3Z0I5OE9Ob2JhQzRPSkJPdDViTXlqaWxwV2JTZnNqS3Z6TExpL256ektmUWdCSEVDS2tYUkhTbXFFNGJ3ZzlrRzRRbWU5OWE3SWQzUjlQWllIb1RmVW02dFBFWDA4bkFpUHF3cURwZGt0OW8rK08yTXhXVlYyMExLVFh3bjhOMm04VDJhQjJONlJUWTczenQwV3FhMGxpVWw2QThwcmxwaElYZmFwQXdXbGVrWVk1S1h1Wk02a1lJdzk4NVdzczdmWmZpZHl3V0VkcWowMVAydVRMMWc2OEE4UXBwWFZLNy0tMU1TSUhXYkR4VHdQZ0tkUUk4Ykordz09--0ba74552e6a1ff0aefa92b641fec5ca3d1d93744; path=/; HttpOnly
Transfer-Encoding:chunked
X-Content-Type-Options:nosniff
X-Frame-Options:SAMEORIGIN
X-Request-Id:dcab253c-1b74-447d-bba2-e5f4f626326e
X-Runtime:1.337000
X-XSS-Protection:1; mode=block
Request Headers
Accept:*/*;q=0.5, text/javascript, application/javascript, application/ecmascript, application/x-ecmascript
Accept-Encoding:gzip, deflate
Accept-Language:it-IT,it;q=0.9,en-US;q=0.8,en;q=0.7
Connection:keep-alive
Content-Length:1311
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Cookie:user_id=BAhpAmwU--b6aed621ab0867a3634a01cf398e861d40a306f4; _shoptools_session=RzhjWjdGOTZuNGV1WXluUnJsSGNDOFV1a1FzdFo1cm15bGExUS8zemhwOW92L1QyR1hkRGJKc25tUkFxSU00K2hYa3JMbFR0V3JHOE1reEExQ0ROb01OS3BlYUxTb3lpcjJLYkxBdW4zQlRzeHFpRmNDbmR1WjIvQ2Y1R2h2WWI4QXh2NituREFiVjJPRmJ4cGN3SUtiOHNEdnlBR3dYVFFOeHBuYjA0YUpsMWVMYW1Nbjg3SWZhSEo0UlUrekl3dHc0UlNNQUpLWkYyL1BQbDU1Y0lPOEZCR3FwSkgzdVBBZUdVWkdoa2g3NmRmdFMxeTh4dU1QeURnZ2hYYVl2bmg4eXdhcUZiRjIyTGxXUEFFQlRaZUw5U3I5cTFDazRFQUFVOWFsc0EwVWRYZm1DdXdUUXhhdFBUbzZzS2dTemxpM05EdnJGOGc4UXZVcmdDRitDbmNxeko2Z3puWUVFeUJleEVQblRvMjZJTVU5UWJYbVd3dVRUNTJ6T3ZoMXZLcTNZcVNLN2wwRVB4a2VzQStkUnMxdVZ4aDhVcEh0N0Urbng4OXFCNWJZMG9ZbTZmK0NiNDJYWWJyTUVQbEFrT2FMS2dMRVVLbUZ2K3JCSzdKc3ZHNkRvM0I4THRKTS91ZkV5TG1tRWxaRzE4b21PWUVQTE5UY3lRZzMrUWNpNEhGR1BvWXAzdU13WGNHWG1wQTRKVXJnYk4xNnRtMXB3QmNwTWxGdjVEcFpGK2Mybjh5eWUwTkIzYzV5c1hTNmptRkJxeENWVm8rN2NZcnoraGMxaUtqSXFkQVcvTDRGLzgyVlM2YUJNLzVEdGVqS1hVQW5VRkJtMXVNaTdqNWZBdGNnL3RYeENTVFVaK2tUS0JMY3Uyb1E9PS0tU01xem9VN2ZKWUE5S0RlUlZGN2hydz09--99e4617fce843b56f953710949378b98d8fd8575
Host:bp-webtools1.otto.boreus.de
Origin:http://bp-webtools1.otto.boreus.de
Referer:http://bp-webtools1.otto.boreus.de/tools/it/reviews/v2/reviews?review_service_host=live
User-Agent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
X-CSRFToken:IErrbmeMS3fR2c7olBjNCRuT3uH9u//1HmRixtccJ0UuB2nFirS1Y8nA/0VMrTBeGvM4ttpTQlCpzmSke/CRfg==
X-Requested-With:XMLHttpRequest
Query String Parameters
review_service_host:live
Form Data
utf8:✓
reviews_review_search_presenter[userID]:5228
reviews_review_search_presenter[reserveYN]:F
reviews_review_search_presenter[timeOffset]:-60
reviews_review_search_presenter[aktiv]:alle
reviews_review_search_presenter[meinungstyp]:R
reviews_review_search_presenter[typus]:
reviews_review_search_presenter[minNote]:1
reviews_review_search_presenter[maxNote]:5
reviews_review_search_presenter[fromDate]:2017-01-01
reviews_review_search_presenter[fromTime]:00:00
reviews_review_search_presenter[toDate]:2017-01-01
reviews_review_search_presenter[toTime]:23:59
reviews_review_search_presenter[fromUpdateDate]:
reviews_review_search_presenter[fromUpdateTime]:
reviews_review_search_presenter[toUpdateDate]:
reviews_review_search_presenter[toUpdateTime]:
reviews_review_search_presenter[artikelNummer]:
reviews_review_search_presenter[masterStyleID]:
reviews_review_search_presenter[digiStyleID]:
reviews_review_search_presenter[id]:
reviews_review_search_presenter[text]:
reviews_review_search_presenter[ownClientOnlyYN]:false
reviews_review_search_presenter[accountNumber]:
reviews_review_search_presenter[name]:
reviews_review_search_presenter[firstName]:
reviews_review_search_presenter[email]:

我试着把这个添加到我的第一个脚本中:

params={
    'reviews_review_search_presenter[userID]': '5228',
    'reviews_review_search_presenter[reserveYN]': 'F',
    'reviews_review_search_presenter[timeOffset]':'-60',
    'reviews_review_search_presenter[aktiv]':'alle',
    'reviews_review_search_presenter[meinungstyp]':'R',
    'reviews_review_search_presenter[typus]': '',
    'reviews_review_search_presenter[minNote]':'1',
    'reviews_review_search_presenter[maxNote]':'5',
    'reviews_review_search_presenter[fromDate]':'2017-01-01',
    'reviews_review_search_presenter[fromTime]':'00:00',
    'reviews_review_search_presenter[toDate]':'2017-01-01',
    'reviews_review_search_presenter[toTime]':'23:59',
    'reviews_review_search_presenter[fromUpdateDate]':'',
    'reviews_review_search_presenter[fromUpdateTime]':'',
    'reviews_review_search_presenter[toUpdateDate]':'',
    'reviews_review_search_presenter[toUpdateTime]':'',
    'reviews_review_search_presenter[artikelNummer]':'',
    'reviews_review_search_presenter[masterStyleID]':'',
    'reviews_review_search_presenter[digiStyleID]':'',
    'reviews_review_search_presenter[id]':'',
    'reviews_review_search_presenter[text]':'',
    'reviews_review_search_presenter[ownClientOnlyYN]':'false',
    'reviews_review_search_presenter[accountNumber]':'',
    'reviews_review_search_presenter[name]':'',
    'reviews_review_search_presenter[firstName]':'',
    'reviews_review_search_presenter[email]':'',
    }
response2 =s.post('http://bp-webtools1.otto.boreus.de/tools/it/reviews/v2/reviews/search_reviews?review_service_host=live', data=params)
print(response2.text)
print(response2.status_code)

但我的状态代码是500。你知道吗

任何关于如何处理表格的想法/提示都将不胜感激(对于刮擦部分,我想我会没事的)。请只是考虑到我不是一个专家,所以任何参考教程,网络资源等也将是有用的。我已经在Stackoverflow上搜索过类似的帖子,但是我找不到任何适合我的问题。 提前谢谢!你知道吗


Tags: textformhttpsearchdeittoolsreview