多线程和代理导致30秒延迟。我该如何避免

1 投票
2 回答
881 浏览
提问于 2025-04-16 12:13

我正在使用PyQt、QThreads和couchdb-python来与本地局域网中的couchDB实例进行通信。

为了测试在网络慢的情况下多线程是否能正常工作,我在图形界面和couch之间放了一个代理。

网络活动被记录下来。日志的开头是

13:52:46.303 (1) send: HEAD /cubic HTTP/1.1CRLFHost: localhost:5981CRLFContent-Length: 0CRLFAccept: application/jsonCRLFUser-Agent: CouchDB-Python/0.8CRLFCRLF

13:52:46.308 (1) recv: HTTP/1.1 200 OKCRLFServer: CouchDB/1.0.1 (Erlang OTP/R13B)CRLFDate: Mon, 21 Feb 2011 13:47:18 GMTCRLFContent-Type: application/jsonCRLFContent-Length: 219CRLFCache-Control: must-revalidateCRLFCRLF
13:53:16.312 (1) recv: link closed
13:53:16.319 (2) send: GET /cubic/_design/Company/_view/by_name HTTP/1.1CRLFHost: localhost:5981CRLFContent-Length: 0CRLFAccept: application/jsonCRLFUser-Agent: CouchDB-Python/0.8CRLFCRLF
13:53:16.330 (2) recv: HTTP/1.1 200 OKCRLFTransfer-Encoding: chunkedCRLFServer: CouchDB/1.0.1 (Erlang OTP/R13B)CRLFEtag: "243QGZGN1ETGA4VN9H1OMB86Z"CRLFDate: Mon, 21 Feb 2011 13:47:48 GMTCRLFContent-Type: application/jsonCRLFCache-Control: must-revalidateCRLFCRLF
13:53:16.331 (2) recv: a0CRLF{"total_rows":9,"offset":0,"rows":[CRLF{"id":"c182c1a2f71c3547ccee45556300770e","key":["A first Company","231","345 East of Eden" ,"Manchester",null],"value":null}CRLF
13:53:16.332 (2) recv: 77CRLF,CRLF{"id":"c182c1a2f71c3547ccee455563001254","key":["C-U-B","1","9-11 March Business Centre","March",null],"value":null}CRLF72CRLF,CRLF{"id":"421261e8a3311c356c4900185800d976","key":"Four","104","4 Fourth street","Four Score",null],"value":null}CRLF74CRLF,CRLF{"id":"c182c1a2f71c3547ccee455563003de8","key":["Fred Smith","431","30 High street","Somehow",null],"value":null}CRLF
13:53:16.334 (2) recv: 83CRLF,CRLF{"id":"7bc7f2014593d108b5b681fe03002ad6","key":["Ian Hobson","1002","31 Sheerwater Dr","Northampton, Nhants",null],"value":null}CRLF76CRLF,CRLF{"id":"c182c1a2f71c3547ccee455563003139","key":["Julie Bloggs","302","30 Back Lane","Somewhere",null],"value":null}CRLF7cCRLF,CRLF{"id":"c182c1a2f71c3547ccee455563002e87","key": "Kingfisher Group","323","33 Westmister Rd","Londonx",null],"value":null}CRLF
13:53:16.335 (2) recv: 7aCRLF,CRLF{"id":"c182c1a2f71c3547ccee455563005894","key":["New Company","212","34 Back Street","Another Town",null],"value":null}CRLF72CRLF,CRLF{"id":"421261e8a3311c356c4900185800d7e2","key":["Three","103","3 High Street","Liverpool 8",null],"value":null}CRLF4CRLFCRLF]}CRLF1CRLFLFCRLF0CRLFCRLF
13:53:46.337 (2) recv: link closed

这正是我预期的,除了在13:52:46.308打开数据库后,有一个30秒的延迟,而接收链接超时。直到这个延迟发生后,代码才能打开视图,这发生在13:53:16.319。

不过,代理使用的是不同的线程{(2)而不是(1)}。没有代理的时候,应用程序部分运行得很好,所以我怀疑是代理的问题。

代理代码是基于microproxy的,下面是代码

import re, sys, datetime
import socket 
import threading 
PORT = 5981
class ConnectionThread(threading.Thread): 
    def __init__(self, (conn,addr),id): 
        self.conn = conn 
        self.addr = addr
        self.id = id
        threading.Thread.__init__(self)

    def report(self,data,dir):
        txt = data.replace("\n",'LF')
        txt = txt.replace("\r",'CR')
        now = datetime.datetime.now()
        print "%s (%s) %s: %s" % (now.isoformat()[11:23],self.id,dir,txt)      

    def run(self): 
        data = self.conn.recv(1024*1024) 
        request = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
        request.connect(('192.168.0.1',5984)) 
        self.report(data,'send')
        request.send(data) 
        while True: 
            temp = request.recv(1024) 
            if ('' == temp):
                self.report('link closed','recv')
                break 
            self.report(temp,'recv')
            self.conn.send(temp)
        self.conn.close()

class ProxyThread(threading.Thread): 
    def __init__(self, port): 
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 
        self.sock.bind(('localhost', port))
        self.count = 1
        threading.Thread.__init__(self) 
    def run(self): 
        self.sock.listen(3) 
        while True:
            temp = ConnectionThread(self.sock.accept(),self.count)
            temp.start()   # each message handled in own thread
            self.count += 1
if __name__ == "__main__": 
    print "Starting a proxy on port", PORT
    proxy = ProxyThread(PORT) 
    proxy.run() 

打开数据库的请求和视图的请求都是在同一个(非图形界面)线程中发出的。为什么会出现延迟呢?

我该如何修改才能避免这种情况呢?

2 个回答

0

根据HTTP规范(第9.4节),

HEAD方法和GET方法是一样的,唯一的区别是服务器在响应中不能返回消息主体。

这就是说,在CRLFCRLF之后没有任何内容,但你的代理正在等待数据,而不是从连接中读取。我怀疑couchdb库试图使用保持连接,但你关注的是错误的套接字。

你的基本问题是,你声称自己支持HTTP 1.1,但实际上并不是。你应该关注CRLFCRLF序列和内容大小头。如果找到这样的序列,就读取那么多数据,然后跳出循环;如果没有找到,就立即跳出。

0

你的循环只从代理连接的一端读取数据。这就导致当服务器的回复被转发给客户端时,如果客户端断开了连接,代码并不会察觉到客户端已经断开,而是继续等待服务器的回复或者断开。

要解决这个问题,需要同时从服务器(请求)和客户端(self.conn)这两个地方读取数据。最简单的方法就是使用非阻塞的套接字。

撰写回答