我该如何通过代码管理celeryd_multi？

Question

到目前为止，我一直在使用 python manage.py celeryd 来启动 Celery，具体是这样启动的：

python manage.py celeryd -l info --settings=settings

在我的视图代码中，做了以下操作：

BinaryExecTask.delay(request.POST["binary_path"])

而在我的 settings.py 文件中的代码是这样的：

import djcelery
djcelery.setup_loader()

BROKER_BACKEND = "djkombu.transport.DatabaseTransport"
#celery
BROKER_HOST = "localhost"
BROKER_PORT = 5672
BROKER_USER = "guest"
BROKER_PASSWORD = "guest"
BROKER_VHOST = "/"

这个设置会在后台执行一些程序。问题是，有些程序运行的时间很短，而有些可能需要半个小时。使用 celeryd 的时候，所有的任务都会被阻塞，直到当前的任务执行完毕。我在这里看到了一些关于如何启动 celeryd_multi 的例子，但当我运行：

python manage.py celeryd_multi start 3 --settings=settings -l info

却出现了这个错误：

celeryd-multi v2.3.1
> Starting nodes...
        > celery1.x: Traceback (most recent call last):
  File "manage.py", line 14, in <module>
    execute_manager(settings)
  File "c:\code\python27\lib\site-packages\django-1.3-py2.7.egg\django\core\management\_
 line 438, in execute_manager
    utility.execute()
  File "c:\code\python27\lib\site-packages\django-1.3-py2.7.egg\django\core\management\_
 line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "c:\code\python27\lib\site-packages\django_celery-2.3.3-py2.7.egg\djcelery\manage
s\celeryd_multi.py", line 22, in run_from_argv
    ["%s %s" % (argv[0], argv[1])] + argv[2:])
  File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
 172, in execute_from_commandline
    self.commands[argv[0]](argv[1:], cmd)
  File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
 205, in start
    retcode = self.waitexec(argv)
  File "c:\code\python27\lib\site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_mul
 354, in waitexec
    pipe = Popen(argstr, env=self.env)
  File "c:\code\python27\lib\subprocess.py", line 672, in __init__
    errread, errwrite)
  File "c:\code\python27\lib\subprocess.py", line 882, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

celeryd-multi start 3 -c 3 也会抛出同样的错误。我该怎么做才能成功启动一个 Celery 实例，让我可以并行运行更多的任务呢？另外，我在视图中需要做什么不同的设置吗？

编辑：一些调试让我找到了这里 ( site-packages\celery-2.3.1-py2.7.egg\celery\bin\celeryd_multi.py(354)waitexec )

351         def waitexec(self, argv, path=sys.executable):
352             args = " ".join([path] + list(argv))
353  ->         argstr = shlex.split(args.encode("utf-8"))
354             pipe = Popen(argstr, env=self.env)
(Pdb) p argstr
['c:codepython27python.exe', 'manage.py', 'celeryd_detach', '-l', 'info', '--pidfile=celeryd@1.pid',
 '-n', 'celery1.x', '--logfile=celeryd@1.log']
(Pdb) p Popen(argstr, env=self.env)
*** WindowsError: WindowsError(2, 'The system cannot find the file specified')
(Pdb)

所以，我们可以看到，Python 的路径被破坏了 :）。接下来我该怎么做呢？

编辑2：我在这里提交了一个问题。

error handling concurrency debugging celery instance management task management background processing parallel execution

我该如何通过代码管理celeryd_multi？

1 个回答

撰写回答