kombu“慢”带内存传输

2024-03-29 00:13:05 发布

您现在位置:Python中文网/ 问答频道 /正文

我想用kombu来释放我的软件的一部分。由于我不需要持久性,也不需要进程间通信,所以我不想使用外部守护进程,所以我尝试了内存内传输。我设置了一些测试代码

from kombu.mixins import ConsumerMixin
from kombu import Connection, Exchange, Queue
from kombu.utils.debug import setup_logging

from time import sleep

import threading

media_exchange = Exchange('media', 'direct')
video_queue = Queue('video', exchange=media_exchange, routing_key='video')
queues = [video_queue]


class Worker(ConsumerMixin, threading.Thread):

    def __init__(self):
        self.connection = Connection('memory:///')

        super(Worker, self).__init__()

    def run(self):
        print("YUP I'M RUNNING")
        super(Worker, self).run()
        print("BUT I WASN'T BLOCKED :(")

    def get_consumers(self, Consumer, channel):
        return [
            Consumer(queues, callbacks=[self.on_message], accept=['json']),
        ]

    def on_message(self, body, message):
        print("RECEIVED MESSAGE: %r" % (body, ))
        message.ack()

if __name__ == '__main__':
    # setup root logger
    setup_logging(loglevel='INFO', loggers=[''])

    try:
        worker = Worker()
        worker.start()
    except KeyboardInterrupt:
        print('bye bye')

    while True:
        print("SEND MESSAGE")
        with Connection('memory:///') as conn:
            producer = conn.Producer(serializer='json')
            producer.publish({"foo": "bar"}, exchange=media_exchange,
                             routing_key='video', declare=queues)
        sleep(0.1)

但是如果我使用内存传输,消息的传递速度很慢,而如果我使用reddis传输,结果似乎还可以。内存示例:

^{pr2}$

reddis示例:

(rpc_video_player)[phas@mir src]$ python test_messages.py 
YUP I'M RUNNING
SEND MESSAGE
Connected to redis://localhost:6379//
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}
SEND MESSAGE
RECEIVED MESSAGE: {'foo': 'bar'}

似乎有一些与缓冲区/预取/队列/内容相关的问题,但我没有真正的线索。我不知道这是配置问题还是内存传输中的错误/限制。任何帮助都是值得的。在


Tags: 内存fromimportselfsendmessageexchangefoo