如何在Python中发布分块编码的数据

Question

我正在尝试将分块编码的数据发布到httpbin.org/POST。我尝试了两种选择：请求和httplib

使用请求

#!/usr/bin/env python

import requests

def gen():
        l = range(130)
        for i in l:
                yield '%d' % i

if __name__ == "__main__":
        url = 'http://httpbin.org/post'
        headers = {
                        'Transfer-encoding':'chunked',
                        'Cache-Control': 'no-cache',
                        'Connection': 'Keep-Alive',
                        #'User-Agent': 'ExpressionEncoder'
                }
        r = requests.post(url, headers = headers, data = gen())
        print r

使用httplib

#!/usr/bin/env python

import httplib
import os.path

if __name__ == "__main__":
        conn = httplib.HTTPConnection('httpbin.org')
        conn.connect()
        conn.putrequest('POST', '/post')
        conn.putheader('Transfer-Encoding', 'chunked')
        conn.putheader('Connection', 'Keep-Alive')
        conn.putheader('Cache-Control', 'no-cache')
        conn.endheaders()
        for i in range(130):
                conn.send(str(i))

        r = conn.getresponse()
        print r.status, r.reason

在这两种情况下，每当我分析Wireshark跟踪时，都不会看到发送多个数据块。相反，我看到的是所有的数据都是在一个块中发送的？我是不是丢了什么东西？

Answer 1

你发布的代码不应该正常工作。您仍然获得成功响应的原因是httpbin.org当前不支持分块传输编码。请参阅错误https://github.com/kennethreitz/httpbin/issues/102。

就像上面链接的postPiotr一样，您应该用十六进制写出每个块的长度，然后写出块本身。

我杀掉了你的代码作为例子。http://httpbin.org/post端点有a form that you can use for testing。这就是我生成chunk1和chunk2表单数据的地方。

import httplib
import time

chunk1 = "custname=bob&custtel=11111&custemail=bob%40email.com&si"
chunk2 = "ze=medium&topping=bacon&delivery=11%3A00&comments=if+you%27re+late+we+get+it+free"

if __name__ == "__main__":
    conn = httplib.HTTPConnection('httpbin.org')
    conn.connect()
    conn.putrequest('POST', '/post')
    conn.putheader('Transfer-Encoding', 'chunked')
    conn.putheader('Content-Type', 'application/x-www-form-urlencoded')
    conn.endheaders()

    conn.send("%s\r\n" % hex(len(chunk1))[2:])
    conn.send("%s\r\n" % chunk1)

    time.sleep(1)

    conn.send("%s\r\n" % hex(len(chunk2))[2:])
    conn.send("%s\r\n" % chunk2)

    time.sleep(1)
    /* last chunk */
    conn.send("0\r\n\r\n")

    r = conn.getresponse()
    print r.status, r.reason, r.read()

wireshark中的流看起来如下所示，这是不正确的，因为它没有等待（注意后面的0）或解释我们发送的请求体（注意json: null）：

POST /post HTTP/1.1
Host: httpbin.org
Accept-Encoding: identity
Transfer-Encoding: chunked
Content-Type: application/x-www-form-urlencoded

37
custname=bob&custtel=11111&custemail=bob%40email.com&si
51
ze=medium&topping=bacon&delivery=11%3A00&comments=if+you%27re+late+we+get+it+free
HTTP/1.1 200 OK
Connection: close
Server: gunicorn/18.0
Date: Fri, 31 Oct 2014 10:37:24 GMT
Content-Type: application/json
Content-Length: 494
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Via: 1.1 vegur

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept-Encoding": "identity", 
    "Connect-Time": "2", 
    "Connection": "close", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "Total-Route-Time": "0", 
    "Transfer-Encoding": "chunked", 
    "Via": "1.1 vegur", 
    "X-Request-Id": "5053a365-ca6a-4c29-b97a-f7a6ded7f2d9"
  }, 
  "json": null, 
  "origin": "110.174.97.16", 
  "url": "http://httpbin.org/post"
}0