如何使用Supervisor啓動Celery時避免SECRET_KEY錯誤？

Question

我有一個運行在Debian 7.8上的Django 1.62應用程序，Nginx 1.2.1作爲我的代理服務器，Gunicorn 19.1.1作爲我的應用程序服務器。我已經安裝了Celery 3.1.7和RabbitMQ 2.8.4來處理異步任務。我試圖使用Supervisor 3.0a8來管理我的各種應用程序，尤其是Celery。問題是，當我嘗試開始通過主管芹菜，我得到這個錯誤：（我顯示在底部的整個堆棧跟蹤）

ImproperlyConfigured: The SECRET_KEY setting must not be empty. 

我所有的配置文件都保存在一個「的conf」目錄只是坐在下面的我‘的Myproj’項目目錄是這樣的：

conf 
├── celeryconfig.py 
├── celeryconfig.pyc 
├── celery.py 
├── __init__.py 
├── middleware.py 
├── settings 
│   ├── base.py 
│   ├── dev.py 
│   ├── __init__.py 
│   ├── prod.py 
├── urls.py 
├── wsgi.py 

我生產的Django的設置都保存在它繼承我base.py基地設置prod.py設置文件。我將密鑰放在我的虛擬環境的postactivate文件中，然後將其讀入我的生產設置中，如下所示。我通過Python解釋器驗證了密鑰存在於我的生產環境中。

# conf/settings/prod.py 
from conf.settings.base import * 
... 
# get_env_variable is defined in base.py 
SECRET_KEY = get_env_variable("SECRET_KEY") 

這是get_env_variable函數讀取密鑰：

# conf/settings/base.py 
def get_env_variable(var_name): 
    try: 
     return os.environ[var_name] 
    except KeyError: 
     error_msg = "Set the %s environment variable" % var_name 
     raise ImproperlyConfigured(error_msg) 

這是我的上司的配置文件。它是基於樣本文件顯示在Celery documentation：

# /etc/supervisor/conf.d/myproj.conf 
[program:myproj] 
command = /www/myproj/bin/start-gunicorn 
user = root 
stdout_logfile = /var/log/gunicorn/supervisor.log 
redirect_stderr = true 

[program:celery] 
directory=/www/myproj 
command=/home/myproj/venv/myproj/bin/celery worker --app=conf -l debug 
user=nobody 
numprocs=1 
stdout_logfile=/var/log/celery/celery.log 
stderr_logfile=/var/log/celery/celery.log 
autostart=true 
autorestart=true 
startsecs=10 
stopwaitsecs=600 
killasgroup=true 
priority=998 

下面是如何使用監事加載新的芹菜配置：

sudo service supervisor stop 
sudo service supervisor start 

這是我的芹菜應用程序文件：

# conf/celery.py 
from __future__ import absolute_import 
import os 
from celery import Celery 
from django.conf import settings 
from conf import celeryconfig 

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'conf.settings') 
app = Celery('conf') 
app.config_from_object(celeryconfig) 
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 

這是我的芹菜配置文件：

# conf/celeryconfig.py 
BROKER_URL = 'amqp://guest@localhost:5672//' 
CELERY_RESULT_BACKEND = 'amqp' 
CELERY_ACCEPT_CONTENT = ['json', ] 
CELERY_TASK_SERIALIZER = 'json' 
CELERY_RESULT_SERIALIZER = 'json' 
CELERY_TASK_RESULT_EXPIRES = 3600 
CELERY_SEND_TASK_ERROR_EMAILS = True 

每芹菜文件，我已經修改了__init__.py：

# conf/__init__.py 
from __future__ import absolute_import 
from .celery import app as celery_app 

我可以用手使用以下命令啓動芹菜，它只是罰款啓動：

workon myproj # Activate project's virtual environment 
celery worker -A conf -l info 

然而，當我嘗試通過主管啓動它，我得到了我所描述的錯誤。考慮到可能Supervisor無法訪問Django密鑰，因爲它是一個環境變量，我試圖在我的prod.py設置文件中對密鑰進行硬編碼（而不是通過get_env_variable函數讀取它），但是這並沒有修復問題。

我嘗試將所有設置合併到包含實際密鑰的prod.py設置文件中，但沒有幫助。

我也嘗試添加該環境參數到超級配置文件，如下所示但這並沒有解決問題：

# /etc/supervisor/conf.d/myproj.conf 
... 
[program:celery] 
environment=SECRET_KEY="(my secret key)" 
directory=/www/myproj 
command=/home/myproj/venv/myproj/bin/celery worker --app=conf -l debug 
user=nobody 
... 

我嘗試了「用戶」設置爲用戶的名字，我啓動網站，但這也沒有幫助。

# /etc/supervisor/conf.d/myproj.conf 
... 
[program:celery] 
environment=SECRET_KEY="(my secret key)" 
directory=/www/myproj 
command=/home/myproj/venv/myproj/bin/celery worker --app=conf -l debug 
user=myproj 
... 

我讀到了一個名爲django-supervisor庫，應該緩解Django和主管之間的集成，但我得到了同樣的錯誤，如果我使用該庫。

最後我讀here on SO如果你的密鑰包含「％」符號，Supervisor不會喜歡它。我注意到我的密鑰確實包含一個「％」符號，所以我像這個「%%」一樣逃脫了它，但是這並沒有解決問題。

任何人都可以看到我做錯了什麼？我在這裏看到了來自不同情況下遇到同樣錯誤的用戶的其他問題，但我試圖實施討論過的各種解決方案，但他們都沒有解決問題。

非常感謝您的想法。對於長期問題抱歉，但這個問題有很多移動部分。

下面是完整的堆棧跟蹤：

Traceback (most recent call last): 
    File "/home/myproj/venv/myproj/bin/celery", line 11, in <module> 
    sys.exit(main()) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/__main__.py", line 30, in main 
    main() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 81, in main 
    cmd.execute_from_commandline(argv) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 769, in execute_from_commandline 
    super(CeleryCommand, self).execute_from_commandline(argv))) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/base.py", line 307, in execute_from_commandline 
    return self.handle_argv(self.prog_name, argv[1:]) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 761, in handle_argv 
    return self.execute(command, argv) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/celery.py", line 693, in execute 
    ).run_from_argv(self.prog_name, argv[1:], command=argv[0]) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/worker.py", line 179, in run_from_argv 
    return self(*args, **options) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/base.py", line 270, in __call__ 
    ret = self.run(*args, **kwargs) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/bin/worker.py", line 212, in run 
    state_db=self.node_format(state_db, hostname), **kwargs 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/worker/__init__.py", line 95, in __init__ 
    self.app.loader.init_worker() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/loaders/base.py", line 128, in init_worker 
    self.import_default_modules() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/loaders/base.py", line 116, in import_default_modules 
    signals.import_modules.send(sender=self.app) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/utils/dispatch/signal.py", line 166, in send 
    response = receiver(signal=self, sender=sender, **named) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/fixups/django.py", line 69, in on_import_modules 
    self.worker_fixup.validate_models() 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/kombu/utils/__init__.py", line 322, in __get__ 
    value = obj.__dict__[self.__name__] = self.__get(obj) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/fixups/django.py", line 64, in worker_fixup 
    self._worker_fixup = DjangoWorkerFixup(self.app) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/celery/fixups/django.py", line 99, in __init__ 
    self._cache = import_module('django.core.cache') 
    File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module 
    __import__(name) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/core/cache/__init__.py", line 69, in <module> 
    if DEFAULT_CACHE_ALIAS not in settings.CACHES: 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/conf/__init__.py", line 54, in __getattr__ 
    self._setup(name) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/conf/__init__.py", line 49, in _setup 
    self._wrapped = Settings(settings_module) 
    File "/home/myproj/venv/myproj/local/lib/python2.7/site-packages/django/conf/__init__.py", line 151, in __init__ 
    raise ImproperlyConfigured("The SECRET_KEY setting must not be empty.") 
django.core.exceptions.ImproperlyConfigured: The SECRET_KEY setting must not be empty. 

Answer 1

在你的芹菜配置，你需要指定要使用的設置。目前你有prod.py您使用爲您的生產代碼，以便您celery.py應該

# conf/celery.py 
from __future__ import absolute_import 
import os 
from celery import Celery 
from django.conf import settings 
from conf import celeryconfig 

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'conf.settings.prod') 
app = Celery('conf') 
app.config_from_object(celeryconfig) 
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 

因此芹菜會知道你想使用（如果你在設置文件夾中有多個設置）的主管