使用urllib2请求与使用发布的JSON（Python 2.7）的build\u opener之间的意外行为

Question

编辑：

找到了解决办法。因为RBC端点是https，所以handler类也需要有https\u请求函数，而不仅仅是http\u请求。你知道吗

Solution for http endpoints

class ChangeTypeProcessor(urllib2.BaseHandler):
    def https_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

    def https_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

---

在python中使用urllib2库发布JSON数据的两种不同方法时，我遇到了奇怪的行为差异。虽然我知道请求是存在的，但我使用的是传统软件，这使得切换更加困难。遗留软件还使用发布数据的build\u opener方法。你知道吗

当试图将JSON发布到RBC的job search时，request对象能够成功地获取JSON数据/响应。然而，OpenerDirector在向RBC的作业库发布JSON时并不能成功获得响应。你知道吗

以下是我用来尝试从RBC获取职务数据的测试仪：

import json
import urllib2
import urlparse

class ChangeTypeProcessor(urllib2.BaseHandler):
    def http_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

data = '{"lang":"en_ca","deviceType":"desktop","country":"ca","ddoKey":"refineSearch","sortBy":"","subsearch":"","from":100,"all_fields":[],"pageName":"search-results","counts":true,"jobs":true,"keywords":"","global":true,"size":50,"sele
cted_fields":null,"sort":null}'
url = "https://jobs.rbc.com/widgets"
#url = "http://httpbin.org/post"
request = urllib2.Request(url)
request.add_header('Content-Type', 'application/json')

response = urllib2.urlopen(request, data)

urlopener = urllib2.build_opener()
urlopener.add_handler(ChangeTypeProcessor())
print(response.read())

connection = urlopener.open(url, data)
print(connection.read())

以下是OpenDirector在发布到站点时产生的错误：

Traceback (most recent call last):
  File "jsonPost.py", line 23, in <module>
    connection = urlopener.open(url, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/local/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 500: Internal Server Error

以下是使用http://httpbin.org/post获得的发布数据：

请求对象的响应：

{
  "args": {},
  "data": "{\"lang\":\"en_ca\",\"deviceType\":\"desktop\",\"country\":\"ca\",\"ddoKey\":\"refineSearch\",\"sortBy\":\"\",\"subsearch\":\"\",\"from\":100,\"all_fields\":[],\"pageName\":\"search-results\",\"counts\":true,\"jobs\":true,\"keywords\":\"\",\"global\":true,\"size\":50,\"selected_fields\":null,\"sort\":null}",
  "files": {},
  "form": {},
  "headers": {
    "Accept-Encoding": "identity",
    "Content-Length": "259",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "Python-urllib/2.7"
  },
  "json": {
    "all_fields": [],
    "country": "ca",
    "counts": true,
    "ddoKey": "refineSearch",
    "deviceType": "desktop",
    "from": 100,
    "global": true,
    "jobs": true,
    "keywords": "",
    "lang": "en_ca",
    "pageName": "search-results",
    "selected_fields": null,
    "size": 50,
    "sort": null,
    "sortBy": "",
    "subsearch": ""
  },
  "origin": "<hidden>",
  "url": "http://httpbin.org/post"
}

OpenDirector的回应：

{
  "args": {},
  "data": "{\"lang\":\"en_ca\",\"deviceType\":\"desktop\",\"country\":\"ca\",\"ddoKey\":\"refineSearch\",\"sortBy\":\"\",\"subsearch\":\"\",\"from\":100,\"all_fields\":[],\"pageName\":\"search-results\",\"counts\":true,\"jobs\":true,\"keywords\":\"\",\"global\":true,\"size\":50,\"selected_fields\":null,\"sort\":null}",
  "files": {},
  "form": {},
  "headers": {
    "Accept-Encoding": "identity",
    "Content-Length": "259",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "Python-urllib/2.7"
  },
  "json": {
    "all_fields": [],
    "country": "ca",
    "counts": true,
    "ddoKey": "refineSearch",
    "deviceType": "desktop",
    "from": 100,
    "global": true,
    "jobs": true,
    "keywords": "",
    "lang": "en_ca",
    "pageName": "search-results",
    "selected_fields": null,
    "size": 50,
    "sort": null,
    "sortBy": "",
    "subsearch": ""
  },
  "origin": "<hidden>",
  "url": "http://httpbin.org/post"
}

值得注意的是，httpbin提供的信息显示所发布的数据完全相同，但由于某些原因，OpenDirector无法获得正确的响应。你知道吗

我想要的解决方案是使OpenDirector成功地获得正确的响应，而无需重新实现代码库以使用请求或请求对象。你知道吗

有人知道是什么导致了这种差异吗？你知道吗

Answer 1

找到了解决办法。因为RBC端点是https，所以handler类也需要有https\u请求函数，而不仅仅是http\u请求。你知道吗

Solution for http endpoints

class ChangeTypeProcessor(urllib2.BaseHandler):
    def https_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req

    def https_request(self, req):
        req.unredirected_hdrs["Content-type"] = "application/json"
        req.unredirected_hdrs["User-Agent"] = "Python-urllib/2.7"
        return req