python eventlet fabric 在运行 celery worker 时出现 IOError: [Errno 1] 操作不允许

Question

我有一个celery工作者，它的任务是远程机器上运行puppet agent，这个过程会阻塞，也就是说它会一直等着这个任务完成才能继续做其他事情。我还有其他的http api流程在不同的任务中，这些任务可以利用eventlet来提高效率。

@app.task(soft_time_limit=600)
def run_puppet_agent(hostname):
    try:
        env.host_string = self.host
        env.user = self.username
        env.password = self.password
        return sudo('puppet agent -t')
    except SoftTimeLimitExceeded:
        raise Exception('Puppet agent TIMED OUT AFTER 600 seconds')
    except Exception as e:
        #run_puppet_agent.retry(args=[hostname], countdown=20)
        LOG.info('')
        raise Exception('Puppet agent failed with error message %s' % e.message)

当我以以下方式运行这个工作者时：

celery multi start 2 -A phantom.celery.manage -P eventlet -c 15 --loglevel=INFO

我遇到了一个异常错误，错误信息如下：

Traceback (most recent call last):
File "/home/uruddarraju/Phantom/phantom/tasks/fabric_tasks.py", line 19, in run_puppet_agent
return sudo(command, quiet=False)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/network.py", line 639, in host_prompting_wrapper
return func(*args, **kwargs)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/operations.py", line 1095, in sudo
stderr=stderr, timeout=timeout, shell_escape=shell_escape,
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/operations.py", line 911, in _run_command
stderr=stderr, timeout=timeout)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/operations.py", line 795, in _execute
worker.raise_if_needed()
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/thread_handling.py", line 12, in wrapper
callable(*args, **kwargs)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/fabric/io.py", line 231, in input_loop
r, w, x = select([sys.stdin], [], [], 0.0)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/eventlet/green/select.py", line 79, in select
listeners.append(hub.add(hub.READ, k, on_read))
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/eventlet/hubs/epolls.py", line 52, in add
self.register(fileno, new=True)
File "/home/uruddarraju/Phantom/tools/virtualenv/local/lib/python2.7/site-packages/eventlet/hubs/poll.py", line 45, in register
self.poll.register(fileno, mask)
IOError: [Errno 1] Operation not permitted

我看过这篇文章：为什么select.select()可以处理磁盘文件但不能处理epoll()？，了解到eventlet在与fabric或其他这种阻塞调用一起使用时会出现问题。我想知道有没有办法告诉celery不要对这个特定的任务进行猴子补丁？

异常处理猴子补丁任务调度 celery fabric eventlet 阻塞调用 puppet agent

python eventlet fabric 在运行 celery worker 时出现 IOError: [Errno 1] 操作不允许

1 个回答

撰写回答