http.client.HTTPException：头部超过100个

Question

因为谷歌没有找到关于错误“http.client.HTTPException: got more than 100 headers”的任何信息，所以我才创建了这个问题。

>>> import http.client as h
>>> conn = h.HTTPConnection("www.coursefinders.com")
>>> conn.request("HEAD","/")
>>> conn.getresponse();
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/http/client.py", line 1148, in getresponse
    response.begin()
  File "/usr/lib/python3.4/http/client.py", line 376, in begin
    self.headers = self.msg = parse_headers(self.fp)
  File "/usr/lib/python3.4/http/client.py", line 267, in parse_headers
    raise HTTPException("got more than %d headers" % _MAXHEADERS)
http.client.HTTPException: got more than 100 headers

这个异常是什么意思，我应该如何正确处理这种错误？在浏览器中网站运行得很好。

Answer 1

在我的Mac电脑上，我把这个加到了我的代码里。

import httplib as http_client

然后我调试这个脚本，找出这个库是从哪里加载的。在我的情况下，它是从这里加载的：

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py 

接着我根据Felix的帖子修改了限制。

sudo vim /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py 

Answer 2

我本来想建议使用 requests 库，但它是基于 http.client 实现的，结果也遇到了同样的问题。为了确认问题是出在库上还是服务器上，我尝试了一个 telnet 会话，结果如下：

Trying 91.250.81.121...
Connected to www.coursefinders.com.
Escape character is '^]'.
HEAD / HTTP\1.1

HTTP/1.1 200 OK
Date: Mon, 14 Apr 2014 08:35:54 GMT
Server: Apache/2.2.16 (Debian)
X-Powered-By: PHP/5.3.3-7+squeeze19
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=2bnr4dpa4e90r2lmbv01smu1b6; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: c_id=496cc5d32486ac8d944e971ad6ec9eb3649ab23cs%3A3%3A%22235%22%3B; expires=Tue, 15-Apr-2014 08:35:54 GMT; path=/
Set-Cookie: login=-1; path=/
Set-Cookie: wc=1; expires=Thu, 09-Apr-2015 08:35:54 GMT
Set-Cookie: login=-1; path=/
Set-Cookie: login=-1; path=/
[... Many Set-Cookie commands omitted ...]
Set-Cookie: login=-1; path=/
Cache-Control: max-age=1, private, must-revalidate
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=utf-8

Connection closed by foreign host.

看起来他们的服务器配置有问题，发出了很多多余的 Set-Cookie 头信息。

似乎没有办法配置 httplib 来接受大量的头信息。我试着找一些不是用 httplib 实现的替代 HTTP 库，但没有找到合适的。

Answer 3

把"C:\Python27\Lib\httplib.py"文件里的"_MAXHEADERS"这个值改成1000或者10000。

Answer 4

这里有一个解决方案，不需要修改库的py文件：

import httplib  # or http.client if you're on Python 3
httplib._MAXHEADERS = 1000

只需把这个放在你代码的最上面就可以了。