Celery worker 是否按照特定顺序检查队列?

1 投票
1 回答
668 浏览
提问于 2025-04-18 21:37

如果一个Celery工作者同时从多个队列中获取任务:

celery -A proj worker -Q foo,bar

它会按照特定的顺序检查这些队列吗?比如说,先检查foo队列,然后再检查bar队列?我希望是这样的。

我一直找不到能明确说明这一点的文档(例如,这里),希望能得到一个更明确的答案。

1 个回答

3

根据一些实验,Celery 似乎并不会按照特定的顺序检查队列(至少在使用 Redis 作为中介的时候):

# demo.py
#
# Usage:
#
#    Add some work to the queues.
#
#        python demo.py add add mul mul add
#
#    Start up Celery:
#
#        celery worker -A demo --queues add,mul

from __future__ import absolute_import

import sys
from celery import Celery

app = Celery('demo', broker  = 'redis://localhost:6379/0')
app.conf.update(CELERY_ACCEPT_CONTENT = ['pickle'])

@app.task
def add(x, y):
    print 'add({}, {}) = {}'.format(x, y, x + y)

@app.task
def mul(x, y):
    print 'mul({}, {}) = {}'.format(x, y, x * y)

def main(qs):
    for q in qs:
        func = add if q == 'add' else mul
        for i in xrange(5, 8):
            func.apply_async((i, i), queue = q)

if __name__ == '__main__':
    main(sys.argv[1:])

示例输出:

# [2014-08-31 13:15:33,005: WARNING/Worker-2] mul(5, 5) = 25
# [2014-08-31 13:15:33,005: WARNING/Worker-8] add(5, 5) = 10
# [2014-08-31 13:15:33,007: WARNING/Worker-4] add(6, 6) = 12
# [2014-08-31 13:15:33,007: WARNING/Worker-3] mul(6, 6) = 36
# [2014-08-31 13:15:33,009: WARNING/Worker-5] add(7, 7) = 14
# [2014-08-31 13:15:33,009: WARNING/Worker-7] mul(7, 7) = 49
# [2014-08-31 13:15:33,011: WARNING/Worker-8] mul(5, 5) = 25
# [2014-08-31 13:15:33,011: WARNING/Worker-1] add(5, 5) = 10
# [2014-08-31 13:15:33,013: WARNING/Worker-4] mul(6, 6) = 36
# [2014-08-31 13:15:33,013: WARNING/Worker-6] add(6, 6) = 12
# [2014-08-31 13:15:33,015: WARNING/Worker-5] mul(7, 7) = 49
# [2014-08-31 13:15:33,015: WARNING/Worker-2] add(7, 7) = 14
# [2014-08-31 13:15:33,016: WARNING/Worker-1] add(5, 5) = 10
# [2014-08-31 13:15:33,017: WARNING/Worker-3] add(6, 6) = 12
# [2014-08-31 13:15:33,018: WARNING/Worker-6] add(7, 7) = 14

撰写回答