Heroku上的Django+芹菜应用程序在本地工作，但部署时工作超时

Question

我有一个使用RabbitMQ的Django+Cellery应用程序，它在本地运行良好（调用heroku local），但在部署时无法执行任务。每次调用任务时，似乎都会出现“工作超时”。组件的版本：Django（3.0.1）、芹菜（4.4.0）

Heroku日志：

2020-01-21T15:38:48.828631+00:00 heroku[router]: at=error code=H12 desc="Request timeout" method=POST path="<REDACTED>" host=<REDACTED> request_id=c59b92be-9b39-4a36-a1d0-e646c9b1f694 fwd="<REDACTED> " dyno=web.1 connect=1ms service=30000ms status=503 bytes=0 protocol=https
2020-01-21T15:38:49.822330+00:00 app[web.1]: [2020-01-21 15:38:49 +0000] [4] [CRITICAL] WORKER TIMEOUT (pid:10)
2020-01-21T15:38:49.823866+00:00 app[web.1]: [2020-01-21 23:38:49 +0800] [10] [INFO] Worker exiting (pid: 10)
2020-01-21T15:38:49.982428+00:00 app[web.1]: [2020-01-21 15:38:49 +0000] [14] [INFO] Booting worker with pid: 14

根据日志，RabbitMQ worker似乎运行正常：

2020-01-21T15:33:53.899791+00:00 app[worker.1]: [2020-01-21 23:33:53,899: INFO/MainProcess] mingle: searching for neighbors
2020-01-21T15:33:54.969851+00:00 app[worker.1]: [2020-01-21 23:33:54,969: INFO/MainProcess] mingle: all alone
2020-01-21T15:33:55.055792+00:00 app[worker.1]: [2020-01-21 23:33:55,055: INFO/MainProcess] celery@aaec950b-24bd-4d29-8c43-c8eeb71e58de ready.

程序文件：

web: gunicorn gotcha.wsgi --log-level debug
worker: celery -A gotcha worker -l info

我已经尝试过通过heroku run python manage.py shell命令在Heroku上运行一个简单的任务（只打印一个字符串），并且该任务似乎没有停止运行

任务：

@task()
def test():
    print("test")

强制退出会导致以下错误：

Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/utils/functional.py", line 43, in __call__
    return self.__value__
AttributeError: 'ChannelPromise' object has no attribute '__value__'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.6/site-packages/amqp/transport.py", line 137, in _connect
    host, port, family, socket.SOCK_STREAM, SOL_TCP)
  File "/app/.heroku/python/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -9] Address family for hostname not supported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/utils/functional.py", line 344, in retry_over_time
    return fun(*args, **kwargs)
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 283, in connect
    return self.connection
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 839, in connection
    self._connection = self._establish_connection()
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 794, in _establish_connection
    conn = self.transport.establish_connection()
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
    conn.connect()
  File "/app/.heroku/python/lib/python3.6/site-packages/amqp/connection.py", line 311, in connect
    self.transport.connect()
  File "/app/.heroku/python/lib/python3.6/site-packages/amqp/transport.py", line 77, in connect
    self._connect(self.host, self.port, self.connect_timeout)
  File "/app/.heroku/python/lib/python3.6/site-packages/amqp/transport.py", line 148, in _connect
    "failed to resolve broker hostname"))
  File "/app/.heroku/python/lib/python3.6/site-packages/amqp/transport.py", line 161, in _connect
    self.sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/app/.heroku/python/lib/python3.6/site-packages/celery/app/task.py", line 425, in delay
    return self.apply_async(args, kwargs)
  File "/app/.heroku/python/lib/python3.6/site-packages/celery/app/task.py", line 568, in apply_async
    **options
  File "/app/.heroku/python/lib/python3.6/site-packages/celery/app/base.py", line 771, in send_task
    amqp.send_task_message(P, name, message, **options)
  File "/app/.heroku/python/lib/python3.6/site-packages/celery/app/amqp.py", line 559, in send_task_message
    **properties
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/messaging.py", line 181, in publish
    exchange_name, declare,
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 518, in _ensured
    return fun(*args, **kwargs)
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/messaging.py", line 187, in _publish
    channel = self.channel
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/messaging.py", line 209, in _get_channel
    channel = self._channel = channel()
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/utils/functional.py", line 45, in __call__
    value = self.__value__ = self.__contract__()
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/messaging.py", line 224, in <lambda>
    channel = ChannelPromise(lambda: connection.default_channel)
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 868, in default_channel
    self.ensure_connection(**conn_opts)
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/connection.py", line 430, in ensure_connection
    callback, timeout=timeout)
  File "/app/.heroku/python/lib/python3.6/site-packages/kombu/utils/functional.py", line 358, in retry_over_time
    sleep(1.0)
KeyboardInterrupt

编辑1： 我正在使用CloudAMQP附加组件。我尝试在本地运行heroku，但这次是使用插件提供的CloudAMQP URL。我第一次运行它时，有一个worker超时，但在随后的尝试中，任务只是没有被接收。因此，这个问题与使用CLoudAMQP有关。我正在使用免费层

只要我将CELERY_BROKER_URL更改为本地托管的RabbitMQ，所有任务似乎都会收到：

12:13:32 PM worker.1 |  [2020-01-22 12:13:32,021: INFO/MainProcess] Received task: accounts.tasks.send_targets_and_codes[fe363114-b058-4486-81ca-cb2bac6cc48c]  ETA:[2020-01-22 12:01:00+08:00] 
12:13:32 PM worker.1 |  [2020-01-22 12:13:32,022: INFO/MainProcess] Received task: accounts.tasks.send_targets_and_codes[1de3aef0-bb10-41cf-b133-0cb3f2b2c34d]  ETA:[2020-01-22 12:06:00+08:00] 
12:13:32 PM worker.1 |  [2020-01-22 12:13:32,023: INFO/MainProcess] Received task: accounts.tasks.send_targets_and_codes[9dd9631d-0bb3-468e-91c0-abfa83ab940e]  ETA:[2020-01-22 12:16:00+08:00] 
12:13:32 PM worker.1 |  [2020-01-22 12:13:32,150: INFO/ForkPoolWorker-1] Task accounts.tasks.send_targets_and_codes[1de3aef0-bb10-41cf-b133-0cb3f2b2c34d] succeeded in 0.024096399996778928s: None
12:13:32 PM worker.1 |  [2020-01-22 12:13:32,150: INFO/ForkPoolWorker-8] Task accounts.tasks.send_targets_and_codes[fe363114-b058-4486-81ca-cb2bac6cc48c] succeeded in 0.024374700005864725s: None
12:13:32 PM worker.1 |  [2020-01-22 12:13:32,186: INFO/ForkPoolWorker-7] Task accounts.tasks.send_targets_and_codes[2c891064-17d8-479b-8849-682889d75d8b] succeeded in 0.06000789999961853s: None

编辑2：事实证明，如果我的本地RabbitMQ服务器正在运行，即使我已将代理URL设置为CloudAMQP one的URL，任务仍将发送到本地服务器。然而，在关闭本地服务器时，我得到了可怕的消息

1:04:48 PM web.1    |  [2020-01-22 13:04:48 +0800] [10964] [CRITICAL] WORKER TIMEOUT (pid:11003)
1:04:48 PM web.1    |  [2020-01-22 13:04:48 +0800] [11003] [INFO] Worker exiting (pid: 11003)
1:04:48 PM web.1    |  [2020-01-22 13:04:48 +0800] [11005] [INFO] Booting worker with pid: 11005

编辑3：我尝试使用本地redis服务器，但发生了完全相同的WORKER TIMEOUT错误。这位工人似乎确实与redis有联系

然而，这里有一件非常不寻常的事情。如果还有一个RabbitMQ本地服务器正在运行，则该服务器将拾取任务（因此不会发生WORKER TIMEOUT错误）。这让我相信芹菜只是被预先配置为RabbitMQ（甚至不是CloudAMQP！）