2013-10-21 132 views
3

工作,當我使用celery + gevent的任務使用subprocess模塊我得到以下堆棧跟蹤:芹菜任務不GEVENT

Traceback (most recent call last): 
    File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 228, in trace_task 
    R = retval = fun(*args, **kwargs) 
    File "/home/venv/admin/lib/python2.7/site-packages/celery/task/trace.py", line 415, in __protected_call__ 
    return self.run(*args, **kwargs) 
    File "/home/webapp/admin/webadmin/apps/loggingquarantine/tasks.py", line 107, in release_mail_task 
    res = call_external_script(popen_obj.communicate) 
    File "/home/webapp/admin/webadmin/apps/core/helpers.py", line 42, in call_external_script 
    return func_to_call(*args, **kwargs) 
    File "/usr/lib64/python2.7/subprocess.py", line 740, in communicate 
    return self._communicate(input) 
    File "/usr/lib64/python2.7/subprocess.py", line 1257, in _communicate 
    stdout, stderr = self._communicate_with_poll(input) 
    File "/usr/lib64/python2.7/subprocess.py", line 1287, in _communicate_with_poll 
    poller = select.poll() 
AttributeError: 'module' object has no attribute 'poll' 

manage.py長相以下(做猴補丁存在):

#!/usr/bin/env python 
from gevent import monkey 
import sys 
import os 

if __name__ == "__main__": 
    if not 'celery' in sys.argv: 
     monkey.patch_all() 
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "webadmin.settings") 
    from django.core.management import execute_from_command_line 
    sys.path.append(".") 
    execute_from_command_line(sys.argv) 

芹菜任務的行爲是否有沒有正確的修補?

p.s.奇怪的是,我在MacOS上的本地設置工作正常,而我在Centos下獲得這樣的例外(所有的軟件包版本都是相同的,初始化和配置腳本)

+0

請參閱http://stackoverflow.com/q/8678307/512251 – user

+0

你用什麼腳本啓動你的工人? –

+0

@btoueg [celeryd](https://gist.github.com/Novarg/7199719#file-celeryd) 和 [的sysconf/celeryd](https://gist.github.com/Novarg/7199701# file-celeryd-sysconf) – Novarg

回答

-1

沒有模擬gevent中的輪詢,所以monkey.patch_all刪除輪詢機制,選擇不模擬:poll,epoll,kqueue,kevent。見gevent.monkey – Make the standard library cooperative

+0

是的,但是'subprocess'模塊實際上檢查'select'模塊中'poll'是否可用。如果它試圖使用'_communicate_with_poll',那麼當'subprocess'被導入的時候'select'沒有被修補,但是如果芹菜通過manage.py啓動並且在那裏修補,怎麼會這樣。 – Novarg