嘿傢伙我正在爲我的學校的一個項目的原型(我是一個研究助理,所以這不是一個分級項目)。我在已經安裝和運行的服務器集羣上運行芹菜(有48個工人/內核)。我的項目的主要內容是我們希望使用芹菜來處理大量的文件/任務。使用Celery(python)&RabbitMQ將任務結果保存到特定文件位置?
Because of this it is very important that we save results to an actual file, we have gigs upon gigs of data and it WON'T fit in RAM while running the traditional task queue/backend.
反正... 我的原型(用一個簡單的add函數):
task.py
from celery import Celery
app=Celery()
@app.task
def mult(x,y):
return x*y
來完成這項工程時,我執行:$ celery worker -A task -l info
但如果我嘗試添加新的後端:
from celery import Celery
app=Celery()
app.conf.update(CELERY_RESULT_BACKEND = 'file://~/Documents/results')
@app.task
def mult(x,y):
return x*y
(我來自柵欄的算法,mathy側前
[2017-08-04 13:22:18,133: CRITICAL/MainProcess] Unrecoverable error:
AttributeError("'NoneType' object has no attribute 'encode'",)
Traceback (most recent call last):
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/kombu/utils/objects.py", line 42, in __get__
return obj.__dict__[self.__name__]
KeyError: 'backend'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/bartolucci/anaconda3/lib/python3.6/site- packages/celery/worker/worker.py", line 203, in start
self.blueprint.start(self)
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/bootsteps.py", line 115, in start
self.on_start()
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/apps/worker.py", line 143, in on_start
self.emit_banner()
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/apps/worker.py", line 158, in emit_banner
' \n', self.startup_info(artlines=not use_image))),
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/apps/worker.py", line 221, in startup_info
results=self.app.backend.as_uri(),
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/kombu/utils/objects.py", line 44, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/app/base.py", line 1183, in backend
return self._get_backend()
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/app/base.py", line 902, in _get_backend
return backend(app=self, url=url)
File "/home/bartolucci/anaconda3/lib/python3.6/site-packages/celery/backends/filesystem.py", line 45, in __init__
self.path = path.encode(encoding)
AttributeError: 'NoneType' object has no attribute 'encode'
我只有2天到這個項目中,還從來沒芹菜(或類似的庫)工作:
我得到一個相當大的錯誤)。我目前正在與芹菜的用戶指南文檔進行爭吵,但他們在這個細節上實際上相當稀疏。
任何幫助非常感謝,並感謝你。
這其實完全沒有回答我的問題。畢竟預計絕對路徑。感謝您的快速回復。 此外,對於有類似於我的問題的任何人,芹菜(默認情況下)可能無法訪問用戶的主目錄。使用/ tmp/results是測試和實驗的好地方。 ''file:/// tmp/results''爲我工作! –