具有异步响应的Twisted http服务器，其中请求必须等待数据变为可用或

Question

我正在尝试编写一个简单的http服务器，用于处理在数据结构中查找响应或超时的异步请求：

1. 请求到达
2. While time<；timeout检查responseCollector的响应（使用requestId作为键）
3. 如果有响应，请返回
4. 如果超时，则返回超时消息

我是twisted的新手，我想知道异步响应的最佳方式是什么。我看了看some twisted Deferred docs和callLater但是我不清楚我到底应该做什么。现在，我运行一个与deferToThread一起使用的阻塞方法，并等待超时时间过去。我得到延迟方法的字符串不可调用错误：

Unhandled error in Deferred:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 497, in __bootstrap
    self.__bootstrap_inner()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 522, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
--- <exception caught here> ---
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/python/threadpool.py", line 210, in _worker
    result = context.call(ctx, function, *args, **kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/python/context.py", line 59, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/twisted/python/context.py", line 37, in callWithContext
    return func(*args,**kw)
exceptions.TypeError: 'str' object is not callable

这是我的代码：

from twisted.web import server, resource
from twisted.internet import reactor, threads
import json
import time

connectedClients = {}
responseCollector = {}

# add fake data to the collector
class FakeData(resource.Resource):
    isLeaf = True

    def render_GET(self, request):
        request.setHeader("content-type", "application/json")
        if 'rid' in request.args and 'data' in request.args:
            rid = request.args['rid'][0]
            data = request.args['data'][0]
            responseCollector[str(rid)] = data
            return json.dumps(responseCollector)
        return "{}"

class RequestHandler(resource.Resource):
    isLeaf = True

    def render_GET(self, request):
        #request.setHeader("content-type", "application/json")

        if not ('rid' in request.args and and 'json' in request.args):
            return '{"success":"false","response":"invalid request"}'

        rid = request.args['rid'][0]
        json = request.args['id'][0]

        # TODO: Wait for data to show up in the responseCollector with same rid
        # as our request without blocking other requests OR timeout
        d = threads.deferToThread(self.blockingMethod(rid))
        d.addCallback(self.ret)
        d.addErrback(self.err)

    def blockingMethod(self,rid):
        timeout  = 5.0
        timeElapsed = 0.0
        while timeElapsed<timeout:
            if rid in responseCollector:
                return responseCollector[rid]
            else:
                timeElapsed+=0.01
                time.sleep(0.01)
        return "timeout"

    def ret(self, hdata):
        return hdata

    def err(self, failure):
        return failure

reactor.listenTCP(8080, server.Site(RequestHandler()))
reactor.listenTCP(9080, server.Site(FakeData()))
reactor.run()

发出请求（当前不返回任何有用的内容）：

http://localhost:8080/?rid=1234&json={%22foo%22:%22bar%22}

添加一些假数据以用于请求：

http://localhost:9080/?rid=1234&data=foo

使用工作版本更新

from twisted.web import server, resource
from twisted.internet import reactor, threads
import json
import time

connectedClients = {}
responseCollector = {}

# add fake data to the collector
class FakeData(resource.Resource):
    isLeaf = True

    def render_GET(self, request):
        request.setHeader("content-type", "application/json")
        if 'rid' in request.args and 'data' in request.args:
            rid = request.args['rid'][0]
            data = request.args['data'][0]
            responseCollector[str(rid)] = data
            return json.dumps(responseCollector)
        return "{}"

class RequestHandler(resource.Resource):
    isLeaf = True

    def render_GET(self, request):

        if not ('rid' in request.args and 'data' in request.args):
            return '{"success":"false","response":"invalid request"}'

        rid = request.args['rid'][0]
        json = request.args['data'][0]

        # TODO: Wait for data to show up in the responseCollector with same rid
        # as our request without blocking other requests OR timeout
        d = threads.deferToThread(self.blockingMethod,rid)
        d.addCallback(self.ret, request)
        d.addErrback(self.err)

        return server.NOT_DONE_YET

    def blockingMethod(self,rid):
        timeout  = 5.0
        timeElapsed = 0.0
        while timeElapsed<timeout:
            if rid in responseCollector:
                return responseCollector[rid]
            else:
                timeElapsed+=0.01
                time.sleep(0.01)
        return "timeout"

    def ret(self, result, request):
        request.write(result)
        request.finish()

    def err(self, failure):
        return failure

reactor.listenTCP(8080, server.Site(RequestHandler()))
reactor.listenTCP(9080, server.Site(FakeData()))
reactor.run()

Answer 1

在render_GET（）中，您应该返回twisted.web.server.NOT_DONE_。您应该将请求对象传递给ret方法：d.addCallback（self.ret，request）

然后在ret（request）中，应该使用request.write（hdata）编写异步数据，并关闭与request.finish（）的连接。

def ret(self, result, request):
    request.write(result)
    request.finish()

一些信息来自：http://twistedmatrix.com/documents/current/web/howto/using-twistedweb.html

Resource rendering occurs when Twisted Web locates a leaf Resource object to handle a web request. A Resource's render method may do various things to produce output which will be sent back to the browser:

• Return a string
• Request a Deferred, return server.NOT_DONE_YET, and call request.write("stuff") and request.finish() later, in a callback on the Deferred.

Answer 2

考虑一下示例中这两个版本的代码行之间的区别：

d = threads.deferToThread(self.blockingMethod(rid))

对

d = threads.deferToThread(self.blockingMethod, rid)

阅读API documentation中的deferToThread，也许还可以阅读一些关于函数对象的Python文档（Python.org文档没有很好地介绍这一点，但是很多第三方文档都有）。