Python Tornado 服务器断开连接

Question

我有一个用Tornado框架搭建的Python HTTP服务器。在处理了几次请求后，服务器就变得不可用。浏览器页面也无法访问。大约过了20秒没有活动后，它又开始工作了。

在10万个请求中，大约有10个异常。在这种负载下，服务器的CPU使用率大约是30%。

为什么服务器会变得不可用呢？

服务器：

start_port = 4400
workers = 1

class MainHandler(tornado.web.RequestHandler):
    def get(self):
        data = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(1000))
        self.write(data)


def server_process(port):
    application = tornado.web.Application([
        (r"/", MainHandler),
    ])

    http_server = tornado.httpserver.HTTPServer(application)

    http_server.listen(port)
    tornado.ioloop.IOLoop.instance().start()


if __name__ == "__main__":
    for i in xrange(workers):
        port = start_port + i
        print 'process started on %d port' % port
        p = Process(target=server_process, args=(port,))
        p.start()

客户端：

def f():
    for i in xrange(500000):
        try:
            r = requests.get('http://127.0.0.1:4400')
            if i % 100 == 0:
                print i, str(r.text)
        except:
            print traceback.format_exc()
            time.sleep(5)


if __name__ == '__main__':
    for j in xrange(1):
        p = Process(target=f)
        p.start()

错误追踪：

Traceback (most recent call last):
  File "/home/me/PycharmProjects/test/client.py", line 16, in f
    if i % 100 == 0:
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 378, in send
    raise ConnectionError(e)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=4400): Max retries exceeded with url: / (Caused by <class 'socket.error'>: [Errno 99] Cannot assign requested address)

更新：

通过实验，我发现每秒300个请求是硬件服务器的极限（不是进程的限制）。这个值和运行的Tornado进程数量无关。添加nginx作为代理服务器也没有帮助。

服务器运行在Ubuntu 12.04和Linux Mint 16上。看起来这个限制和Debian操作系统有关。

异常处理代理服务器 http服务器 cpu使用率操作系统限制服务器性能 tornado 请求负载

Python Tornado 服务器断开连接

1 个回答

撰写回答