用subprocess.Popen进行twisted线程操作？

Question

我正在尝试用Twisted实现一个服务，这个服务和这里的“finger”教程很相似：http://twistedmatrix.com/documents/current/core/howto/tutorial/intro.html

我有一个基本的.LineListener在等待命令并执行它，然后有一个客户端连接并发出命令。问题是，有时候这个命令需要执行其他东西，我用的是Python的subprocess模块来处理这个。问题不仅仅是调用communicate()时会卡住，这个是正常的subprocess问题，我知道怎么解决。更麻烦的是，subprocess.Popen的调用也会卡住。

下面是Twisted服务器的代码：

from twisted.application import internet, service
from twisted.internet import protocol, reactor, defer, threads
from twisted.protocols import basic
import sys
import time
import subprocess

class MyProtocol(basic.LineReceiver):
    def lineReceived(self, line):
        self.go()
    def go(self):
        def writeResponse(message):
            self.transport.write(message + '\r\n')
            self.transport.loseConnection()
        threads.deferToThread(self.factory.action).addCallback(writeResponse)
    def connectionMade(self):
        self.lines = []

class ActionService(service.Service):
    def __init__(self, **kwargs):
        pass
        #self.users = kwargs

def action(self):
    print "launching subprocess"
    sys.stdout.flush()
    p = subprocess.Popen(["ls"], stderr=subprocess.PIPE, stdout=subprocess.PIPE, stdin=subprocess.PIPE)
    print "launched subprocess, trying to communicate..."
    sys.stdout.flush()
    p.communicate()
    print "returning"
    sys.stdout.flush()
    return "%032d" % (0)

    def getActionFactory(self):
        f = protocol.ServerFactory()
        f.protocol = MyProtocol
        f.action = self.action
        return f

reactor.suggestThreadPoolSize(300)
application = service.Application('Action', uid=0, gid=0)
f = ActionService()
serviceCollection = service.IServiceCollection(application)
internet.TCPServer(31337,f.getActionFactory()
                   ).setServiceParent(serviceCollection)

...这是一些客户端的代码：

#!/usr/bin/python
import time
import threading
import socket

def connectAction(host):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((host, 31337))
    s.send("asdf\r\n")
    resp = s.recv(32)
    s.close()
    return resp

class sscceThread(threading.Thread):
    def __init__(self, host):
        self.host = host
        threading.Thread.__init__(self)
    def run(self):
        connectAction(self.host)

def main():
    threads = []
    for i in range(0, 1000):
        for j in range(0,5):
            t = sscceThread("localhost")
            t.start()
            threads.append(t)
        for t in threads:
            t.join()
        print i
        time.sleep(1)
    #    print i

if __name__ == "__main__":
    main()

通过运行以下命令来启动服务：

twistd -y sscce_twisted_service.py -l twistdLog; tail -f twistdLog

然后通过运行以下命令来启动客户端：

./sscce_twisted_client.py

你应该能看到客户端运行几次（我见过最多运行10次），然后就卡住了。客户端代码中有一个1秒的延迟，这样你可以分辨出每次迭代的Twisted日志条目，而在卡住的那次，你会在Twisted日志中看到类似这样的内容：

2009-12-22 11:18:47-0800 [MyProtocol,55,127.0.0.1] launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,56,127.0.0.1] launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,55,127.0.0.1] launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,57,127.0.0.1]  launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,58,127.0.0.1] launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,56,127.0.0.1] launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,55,127.0.0.1] returning
2009-12-22 11:18:47-0800 [MyProtocol,57,127.0.0.1]  launching subprocess 
2009-12-22 11:18:47-0800 [MyProtocol,56,127.0.0.1]  launching subprocess returning
2009-12-22 11:18:47-0800 [MyProtocol,59,127.0.0.1]  launching subprocess
2009-12-22 11:18:47-0800 [MyProtocol,58,127.0.0.1]  launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,58,127.0.0.1] returning
2009-12-22 11:18:47-0800 [MyProtocol,59,127.0.0.1] launched subprocess, trying to communicate...
2009-12-22 11:18:47-0800 [MyProtocol,59,127.0.0.1] returning

特别需要注意的是MyProtocol,57。它说它准备启动子进程，但从未打印出“启动子进程，尝试通信”的那一行。我觉得它可能就在这里卡住了。

subprocess command execution threading network programming process management twisted asynchronous event-driven

用subprocess.Popen进行twisted线程操作？

1 个回答

撰写回答