多处理工具包
pistil的Python项目详细描述
简单的多处理工具包。这是基于Gunicorn多处理引擎的。
此库允许您监视多种类型的工人和链 主管。优雅，重新加载，工人之间的信号处理。
启动一个工人的简单仲裁者：
from pistil.arbiter import Arbiter
from pistil.worker import Worker
class MyWorker(Worker):
def handle(self):
print "hello from worker n°%s" % self.pid
if __name__ == "__main__":
conf = {}
specs = [(MyWorker, 30, "worker", {}, "test")]
a = Arbiter(conf, specs)
a.run()
同池仲裁者不同。这次我们把 3个操作系统进程上的同一个工作进程：
from pistil.pool import PoolArbiter
from pistil.worker import Worker
class MyWorker(Worker):
def handle(self):
print "hello from worker n°%s" % self.pid
if __name__ == "__main__":
conf = {"num_workers": 3 }
spec = (MyWorker, 30, "worker", {}, "test",)
a = PoolArbiter(conf, spec)
a.run()
该模式的一个常见用法是tcp服务器tjhat共享相同的 它们之间的插座。为此目的，雌蕊提供Tcparbiter和 要与gevent一起使用的tcpsyncworker和geventtcpworker。
雌蕊允许你在仲裁者中混合不同种类的工人：
from pistil.arbiter import Arbiter
from pistil.worker import Worker
class MyWorker(Worker):
def handle(self):
print "hello worker 1 from %s" % self.name
class MyWorker2(Worker):
def handle(self):
print "hello worker 2 from %s" % self.name
if __name__ == '__main__':
conf = {}
specs = [
(MyWorker, 30, "worker", {}, "w1"),
(MyWorker2, 30, "worker", {}, "w2"),
(MyWorker2, 30, "kill", {}, "w3")
]
# launchh the arbiter
arbiter = Arbiter(conf, specs)
arbiter.run()
您还可以连锁仲裁员：
import time
import urllib2
from pistil.arbiter import Arbiter
from pistil.worker import Worker
from pistil.tcp.sync_worker import TcpSyncWorker
from pistil.tcp.arbiter import TcpArbiter
from http_parser.http import HttpStream
from http_parser.reader import SocketReader
class MyTcpWorker(TcpSyncWorker):
def handle(self, sock, addr):
p = HttpStream(SocketReader(sock))
path = p.path()
data = "welcome wold"
sock.send("".join(["HTTP/1.1 200 OK\r\n",
"Content-Type: text/html\r\n",
"Content-Length:" + str(len(data)) + "\r\n",
"Connection: close\r\n\r\n",
data]))
class UrlWorker(Worker):
def handle(self):
f = urllib2.urlopen("http://localhost:5000")
print f.read()
class MyPoolArbiter(TcpArbiter):
def on_init(self, conf):
TcpArbiter.on_init(self, conf)
# we return a spec
return (MyTcpWorker, 30, "worker", {}, "http_welcome",)
if __name__ == '__main__':
conf = {"num_workers": 3, "address": ("127.0.0.1", 5000)}
specs = [
(MyPoolArbiter, 30, "supervisor", {}, "tcp_pool"),
(UrlWorker, 30, "worker", {}, "grabber")
]
arbiter = Arbiter(conf, specs)
arbiter.run()
此示例在端口5000和另一个端口上启动一个具有3个工作线程的Web服务器 正在获取此服务器承载的欢迎页的工作进程：
$ python examples/multiworker2.py 2011-08-08 00:05:42 [13195] [DEBUG] Arbiter master booted on 13195 2011-08-08 00:05:42 [13196] [INFO] Booting grabber (worker) with pid: 13196 ici 2011-08-08 00:05:42 [13197] [INFO] Booting pool (supervisor) with pid: 13197 2011-08-08 00:05:42 [13197] [DEBUG] Arbiter pool booted on 13197 2011-08-08 00:05:42 [13197] [INFO] Listening at: http://127.0.0.1:5000 (13197) 2011-08-08 00:05:42 [13198] [INFO] Booting worker (worker) with pid: 13198 2011-08-08 00:05:42 [13199] [INFO] Booting worker (worker) with pid: 13199 welcome world welcome world
更多的文件正在提交。参见示例中的示例/ 文件夹。
欢迎加入QQ群-->： 979659372
