如果没有Python或Ruby上的selenium，如何查询结果

Question

我在跟踪机票的趋势

目前，我正在使用selenium来模拟提交表单。你知道吗

如你所知，硒是缓慢的，消耗更多的记忆。你知道吗

但是，当您提交表单时，它会将您重定向到一个新的urlhttp://makeabooking.flyscoot.com/Flight/Select

因此，我不知道没有硒我怎么能做到这一点。你知道吗

因为我无法更改这样的查询形式http://makeabooking.flyscoot.com/Flight/from={TPE}&to={NYK}&date={2015-10-12}来获取结果。你知道吗

有没有想过用Ruby或者Python用SSL代理和HTTP代理来实现这一点？你知道吗

示例网站：http://www.flyscoot.com/index.php/en/

Answer 1

我想这个答案https://stackoverflow.com/a/1196151/1033953就是你想要的。你知道吗

您需要检查该表单上的参数以确保发布的值正确，但是您只需要使用Ruby net/http发送httppost。你知道吗

我相信Python也有类似的东西。或者您可以使用curl来发布这个答案https://superuser.com/a/149335

Answer 2

您可以轻松地从chrome获取curl请求，并通过以下方式使用它：

F12 > Network > request > Right Click > Copy As cURL 

curl 'http://makeabooking.flyscoot.com/Flight/Select' -H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8,tr;q=0.6' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8' -H 'Referer: http://www.flyscoot.com/index.php/en/' -H 'Cookie: optimizelyEndUserId=oeu1444666692081r0.12463579000905156; __utmt=1; granify.lasts@1345=1444666699786; ASP.NET_SessionId=lql5yzv1l3yatkh1lcumg2e5; dotrez=1209262602.20480.0000; optimizelySegments=%7B%222335550040%22%3A%22gc%22%2C%222344180004%22%3A%22referral%22%2C%222354350067%22%3A%22false%22%2C%222355380121%22%3A%22none%22%7D; optimizelyBuckets=%7B%223025070068%22%3A%223020800213%22%7D; __utma=185425846.733949751.1444666694.1444666694.1444666694.1; __utmb=185425846.2.10.1444666694; __utmc=185425846; __utmz=185425846.1444666694.1.1.utmcsr=stackoverflow.com|utmccn=(referral)|utmcmd=referral|utmcct=/questions/33084039/how-could-i-query-the-result-without-selenium-on-python-or-ruby; granify.uuid=68b0d8e8-d068-40d8-9068-3098e870b858; granify.session@1345=1444666699786; granify.flags@1345=8; _gr_ep_sent=1; _gr_er_sent=1; granify.session_init@1345=2; optimizelyPendingLogEvents=%5B%5D' -H 'Connection: keep-alive' -H 'X-FirePHP-Version: 0.0.6' -H 'Cache-Control: max-age=0' compressed

如果可以正确设置headers和cookies信息，就可以使用Python请求。如果要将其转换为Python请求，可以使用thislink。这样你就可以模拟浏览器了。参见pyton请求：

cookies = {
    'optimizelyEndUserId': 'oeu1444666692081r0.12463579000905156',
    '__utmt': '1',
    'granify.lasts@1345': '1444666699786',
    'ASP.NET_SessionId': 'lql5yzv1l3yatkh1lcumg2e5',
    'dotrez': '1209262602.20480.0000',
    'optimizelySegments': '%7B%222335550040%22%3A%22gc%22%2C%222344180004%22%3A%22referral%22%2C%222354350067%22%3A%22false%22%2C%222355380121%22%3A%22none%22%7D',
    'optimizelyBuckets': '%7B%223025070068%22%3A%223020800213%22%7D',
    '__utma': '185425846.733949751.1444666694.1444666694.1444666694.1',
    '__utmb': '185425846.2.10.1444666694',
    '__utmc': '185425846',
    '__utmz': '185425846.1444666694.1.1.utmcsr=stackoverflow.com|utmccn=(referral)|utmcmd=referral|utmcct=/questions/33084039/how-could-i-query-the-result-without-selenium-on-python-or-ruby',
    'granify.uuid': '68b0d8e8-d068-40d8-9068-3098e870b858',
    'granify.session@1345': '1444666699786',
    'granify.flags@1345': '8',
    '_gr_ep_sent': '1',
    '_gr_er_sent': '1',
    'granify.session_init@1345': '2',
    'optimizelyPendingLogEvents': '%5B%5D',
}

headers = {
    'Accept-Encoding': 'gzip, deflate, sdch',
    'Accept-Language': 'en-US,en;q=0.8,tr;q=0.6',
    'Upgrade-Insecure-Requests': '1',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
    'Referer': 'http://www.flyscoot.com/index.php/en/',
    'Connection': 'keep-alive',
    'X-FirePHP-Version': '0.0.6',
    'Cache-Control': 'max-age=0',
}

requests.get('http://makeabooking.flyscoot.com/Flight/Select', headers=headers, cookies=cookies)

如果保存结果，则可以通过浏览器（打开）看到结果堆栈.html)地址：

r = requests.get('http://makeabooking.flyscoot.com/Flight/Select', headers=headers, cookies=cookies
f = open("stack1.html", "w")
f.write(r.content)