Celery worker 是否按照特定顺序检查队列?
如果一个Celery工作者同时从多个队列中获取任务:
celery -A proj worker -Q foo,bar
它会按照特定的顺序检查这些队列吗?比如说,先检查foo
队列,然后再检查bar
队列?我希望是这样的。
我一直找不到能明确说明这一点的文档(例如,这里),希望能得到一个更明确的答案。
1 个回答
3
根据一些实验,Celery 似乎并不会按照特定的顺序检查队列(至少在使用 Redis 作为中介的时候):
# demo.py
#
# Usage:
#
# Add some work to the queues.
#
# python demo.py add add mul mul add
#
# Start up Celery:
#
# celery worker -A demo --queues add,mul
from __future__ import absolute_import
import sys
from celery import Celery
app = Celery('demo', broker = 'redis://localhost:6379/0')
app.conf.update(CELERY_ACCEPT_CONTENT = ['pickle'])
@app.task
def add(x, y):
print 'add({}, {}) = {}'.format(x, y, x + y)
@app.task
def mul(x, y):
print 'mul({}, {}) = {}'.format(x, y, x * y)
def main(qs):
for q in qs:
func = add if q == 'add' else mul
for i in xrange(5, 8):
func.apply_async((i, i), queue = q)
if __name__ == '__main__':
main(sys.argv[1:])
示例输出:
# [2014-08-31 13:15:33,005: WARNING/Worker-2] mul(5, 5) = 25
# [2014-08-31 13:15:33,005: WARNING/Worker-8] add(5, 5) = 10
# [2014-08-31 13:15:33,007: WARNING/Worker-4] add(6, 6) = 12
# [2014-08-31 13:15:33,007: WARNING/Worker-3] mul(6, 6) = 36
# [2014-08-31 13:15:33,009: WARNING/Worker-5] add(7, 7) = 14
# [2014-08-31 13:15:33,009: WARNING/Worker-7] mul(7, 7) = 49
# [2014-08-31 13:15:33,011: WARNING/Worker-8] mul(5, 5) = 25
# [2014-08-31 13:15:33,011: WARNING/Worker-1] add(5, 5) = 10
# [2014-08-31 13:15:33,013: WARNING/Worker-4] mul(6, 6) = 36
# [2014-08-31 13:15:33,013: WARNING/Worker-6] add(6, 6) = 12
# [2014-08-31 13:15:33,015: WARNING/Worker-5] mul(7, 7) = 49
# [2014-08-31 13:15:33,015: WARNING/Worker-2] add(7, 7) = 14
# [2014-08-31 13:15:33,016: WARNING/Worker-1] add(5, 5) = 10
# [2014-08-31 13:15:33,017: WARNING/Worker-3] add(6, 6) = 12
# [2014-08-31 13:15:33,018: WARNING/Worker-6] add(7, 7) = 14