首页 / 问题 / 正文

在Pyramid中使用Celery和SQLAlchemy

5 投票
1 回答
3108 浏览
提问于 2025-04-17 06:07

我正在用Pyramid-1.2.1创建一个网页应用，数据库用的是SQLAlchemy。
现在我需要在这个应用里做一些定期任务，我想用Celery作为任务处理器，同时用SQLAlchemy作为消息队列和结果存储。

1) 我已经安装了Celery-2.1.4，但我不知道怎么让它工作。这里是我的 celeryconfig.py 文件：

# List of modules to import when celery starts.
CELERY_IMPORTS = ("my_app.tasks", )

## Result store settings.
CELERY_RESULT_BACKEND = "database"
CELERY_RESULT_DBURI = "sqlite:///MyDataBase.db"
CELERY_RESULT_SERIALIZER = "json"

## Broker settings.
BROKER_TRANSPORT = "sqlakombu.transport.Transport"
BROKER_HOST = "sqlite:///MyDataBase.db"

在 my_app.tasks 里有一个简单的加法任务，来自Celery的示例。
现在当我运行

$ celeryd -l info

我看到以下内容：

[2011-11-11 20:22:50,750: WARNING/MainProcess] celery@OrangeMac.local v2.1.4 is starting.
[2011-11-11 20:22:50,765: WARNING/MainProcess]  
Configuration ->
    . broker -> sqlakombu.transport.Transport://guest@sqlite:///MyDataBase.db/
    . queues ->
        . celery -> exchange:celery (direct) binding:celery
    . concurrency -> 4
    . loader -> celery.loaders.default.Loader
    . logfile -> [stderr]@INFO
    . events -> OFF
    . beat -> OFF
    . tasks ->
        . chatrooms.task.add
[2011-11-11 20:22:50,787: INFO/PoolWorker-1] child process calling self.run()
[2011-11-11 20:22:50,789: INFO/PoolWorker-2] child process calling self.run()
[2011-11-11 20:22:50,791: INFO/PoolWorker-3] child process calling self.run()
[2011-11-11 20:22:50,796: INFO/PoolWorker-4] child process calling self.run()
[2011-11-11 20:22:50,802: WARNING/MainProcess] celery@OrangeMac.local has started.
[2011-11-11 20:22:50,804: WARNING/MainProcess] Traceback (most recent call last):
[2011-11-11 20:22:50,805: WARNING/MainProcess] File "/Users/shashkin/python_v_env/bin/celeryd", line 8, in <module>
[2011-11-11 20:22:50,805: WARNING/MainProcess] load_entry_point('celery==2.1.4', 'console_scripts', 'celeryd')()
[2011-11-11 20:22:50,805: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/bin/celeryd.py", line 166, in main
[2011-11-11 20:22:50,805: WARNING/MainProcess] worker.execute_from_commandline()
[2011-11-11 20:22:50,806: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/bin/base.py", line 40, in execute_from_commandline
[2011-11-11 20:22:50,806: WARNING/MainProcess] return self.run(*args, **vars(options))
[2011-11-11 20:22:50,806: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/bin/celeryd.py", line 85, in run
[2011-11-11 20:22:50,806: WARNING/MainProcess] return Worker(**kwargs).run()
[2011-11-11 20:22:50,806: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/apps/worker.py", line 121, in run
[2011-11-11 20:22:50,807: WARNING/MainProcess] self.run_worker()
[2011-11-11 20:22:50,807: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/apps/worker.py", line 219, in run_worker
[2011-11-11 20:22:50,807: WARNING/MainProcess] worker.start()
[2011-11-11 20:22:50,807: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/__init__.py", line 217, in start
[2011-11-11 20:22:50,808: WARNING/MainProcess] component.start()
[2011-11-11 20:22:50,808: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 238, in start
[2011-11-11 20:22:50,808: WARNING/MainProcess] self.reset_connection()
[2011-11-11 20:22:50,808: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 416, in reset_connection
[2011-11-11 20:22:50,808: WARNING/MainProcess] self.connection = self._open_connection()
[2011-11-11 20:22:50,808: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 480, in _open_connection
[2011-11-11 20:22:50,809: WARNING/MainProcess] max_retries=conf.BROKER_CONNECTION_MAX_RETRIES)
[2011-11-11 20:22:50,809: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/utils/__init__.py", line 276, in retry_over_time
[2011-11-11 20:22:50,809: WARNING/MainProcess] retval = fun(*args, **kwargs)
[2011-11-11 20:22:50,809: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/celery-2.1.4-py2.7.egg/celery/worker/listener.py", line 472, in _establish_connection
[2011-11-11 20:22:50,809: WARNING/MainProcess] conn.connect()                              # evaluate connection
[2011-11-11 20:22:50,809: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 170, in connect
[2011-11-11 20:22:50,810: WARNING/MainProcess] return self.connection
[2011-11-11 20:22:50,810: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 135, in connection
[2011-11-11 20:22:50,810: WARNING/MainProcess] self._connection = self._establish_connection()
[2011-11-11 20:22:50,810: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 148, in _establish_connection
[2011-11-11 20:22:50,810: WARNING/MainProcess] return self.create_backend().establish_connection()
[2011-11-11 20:22:50,810: WARNING/MainProcess] File "/Users/shashkin/python_v_env/lib/python2.7/site-packages/carrot-0.10.7-py2.7.egg/carrot/connection.py", line 161, in create_backend
[2011-11-11 20:22:50,810: WARNING/MainProcess] return backend_cls(connection=self)
[2011-11-11 20:22:50,811: WARNING/MainProcess] TypeError
[2011-11-11 20:22:50,811: WARNING/MainProcess] :
[2011-11-11 20:22:50,811: WARNING/MainProcess] __init__() takes exactly 2 arguments (1 given)
[2011-11-11 20:22:50,811: INFO/MainProcess] process shutting down

我缺少了什么呢？

2) 之后我想在我的应用里启动Celery。安装了celery-pylons包后，我该怎么做？我需要把Celery的设置放在应用的 .ini 文件里吗？能不能给我一个例子，因为我在网上找到的那些对我帮助不大。

谢谢。

数据库管理 sqlalchemy web 应用 celery 消息队列 pyramid 定期任务 任务处理器

1 个回答

4

这看起来是因为celery包和它的依赖项之间的版本不兼容导致的问题。使用最新版本的celery，2.4.5，你的例子可以正常工作。但是如果使用2.1.4版本，我就会遇到同样的错误。

如果你出于某种原因只需要使用celery的2.1.4版本，你就得找出哪些依赖项需要降级才能让它正常工作。从错误信息来看，最有可能需要降级的是carrot这个依赖，但如果其他依赖又需要特定版本的carrot，你可能会陷入依赖地狱。

我强烈建议使用虚拟环境（virtualenv）和pip，这样你可以更轻松地尝试不同版本的包。直接使用Linux软件库中的Python包进行开发可能会非常麻烦。

buildout版本或pip需求可以解决这类问题，但据我所知，celery并没有一套固定的依赖版本。

回答于 2025-04-17 由 Python大师
分享 举报

撰写回答