2
我在一個應用程序中運行Celery 3.0.24和Celery-with-redis 3.0，該應用程序使用週期性任務來輪詢json飼料，並根據它自動執行任務。芹菜定期任務獲取太多次
芹菜正確地宣佈每1分鐘發生一次應有的任務。但是，看起來該任務正在拾取2-3次，這導致應用程序背後的應用程序重複並重復運行三次。
這個問題通常不會在一天或一週內發生，但隨後會開始，即使停止並重新啓動應用程序也不會消失。
詳情：
- 與主管運行它
- 隊列是Redis的（未RabbitMQ的）
當程序運行時，過程（PS AUX風格）顯示爲這樣：
[[email protected]:MainProcess] -active- (worker --config celeryconfig --beat --schedule=/mnt/services/my_app/var/lib/celerybeat --loglevel=INFO --autoreload --app=my_app.queue.tasks --events --queues=my_app)
celerybeat.conf：
[program:celerybeat]
command=/mnt/services/my_app/bin/celery worker --config celeryconfig --beat --schedule=/mnt/services/my_app/var/lib/celerybeat --loglevel=INFO --autoreload --app=my_app.queue.tasks --events --queues=my_app
environment=PYTHONPATH=/mnt/services/my_app/conf
autostart=false
autorestart=true
startsecs=5
startretries=1
stopwaitsecs=300
numprocs=1
stopsignal=TERM
killasgroup=true
stdout_logfile=/mnt/services/my_app/var/log/celerybeat.log
stderr_logfile=/mnt/services/my_app/var/log/celerybeat.err
tasks.py包含此任務：
@periodic_task(
run_every=datetime.timedelta(seconds=60),
name='tasks.my_app_fetch_and_parse_feed',
max_retries=0,
queue='my_app',
options={'queue': 'my_app'},
)
def my_app_fetch_and_parse_feed():
"""
Runs every minute and checks for
matches that need notifications sent.
"""
# some code that's getting run multiple times
pass
任何幫助和提示，任何人都可以給這個將不勝感激！我已經通過我所有關於如何解決這個問題的想法排除了故障。非常感謝！
-
-
- 新增信息 - - -
-
有關芹菜的過程是：
$ ps xuf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
507 29554 0.0 0.0 12768 4828 pts/4 S+ 2013 0:00 -bash
507 22921 0.0 0.0 12920 5408 pts/0 S 19:22 0:00 -bash
507 25582 0.0 0.0 8584 812 pts/0 R+ 19:41 0:00 \_ ps xuf
507 13968 0.0 0.0 12804 5376 pts/1 S+ Feb04 0:00 -bash
507 7617 0.0 0.1 106536 12284 ? Ss 2013 60:06 python2.7 /mnt/services/my_app/bin/supervisord
507 23894 13.0 0.3 154644 25168 ? Rl 19:29 1:32 \_ [[email protected]:MainProcess] -active- (worker --beat --schedule=/mnt/services/my_app/var/lib/celerybeat --loglevel=INFO --autoreload --app=my_app
507 23901 0.0 0.2 147852 22608 ? S 19:29 0:00 \_ [celerybeat]
507 23902 6.2 0.3 143476 26288 ? S 19:29 0:44 \_ [[email protected]:PoolWorker-2]
507 23903 6.3 0.3 143980 26712 ? S 19:29 0:44 \_ [[email protected]:PoolWorker-3]
還是爲了更詳細的所有與芹菜包裝盒上的過程的輸出：
$ ps aux | grep celery
APP_TWO 22229 0.0 0.3 164808 26244 ? Sl 2013 2:01 python2.6 /mnt/services/APP_TWO-john/bin/celeryd --loglevel=INFO --autoreload -A APP_TWO.queue.tasks -E
APP_TWO 22266 0.0 0.3 153868 25536 ? S 2013 2:08 python2.6 /mnt/services/APP_TWO-john/bin/celeryd --loglevel=INFO --autoreload -A APP_TWO.queue.tasks -E
APP_TWO 22267 0.0 0.3 153312 24112 ? S 2013 2:05 python2.6 /mnt/services/APP_TWO-john/bin/celeryd --loglevel=INFO --autoreload -A APP_TWO.queue.tasks -E
APP_TWO 22000 0.0 0.0 3820 528 pts/2 S+ 2013 0:01 tail -n 30 -F var/log/celeryd.err
APP_FOUR 22055 0.0 0.0 3820 516 pts/3 S+ 2013 0:00 tail -F var/log/celeryd.err
APP_TWO 12190 0.0 0.3 159004 24316 ? Sl Jan06 0:53 python2.6 /mnt/services/APP_TWO/bin/celeryd --loglevel=INFO --autoreload -A APP_TWO.queue.tasks -E -Q APP_TWO
APP_TWO 12212 0.0 0.2 140736 20252 ? S Jan06 0:39 python2.6 /mnt/services/APP_TWO/bin/celeryd --loglevel=INFO --autoreload -A APP_TWO.queue.tasks -E -Q APP_TWO
APP_TWO 12215 0.0 0.2 140760 20260 ? S Jan06 0:48 python2.6 /mnt/services/APP_TWO/bin/celeryd --loglevel=INFO --autoreload -A APP_TWO.queue.tasks -E -Q APP_TWO
flume-ng 27903 0.0 0.0 3816 524 ? S Jan24 0:00 tail -F /mnt/services/APP_TWO/var/log/celeryd.err
flume-ng 27904 0.0 0.0 3816 524 ? S Jan24 0:00 tail -F /mnt/services/APP_FOUR/var/log/celeryd.log
flume-ng 27927 0.0 0.0 3820 576 ? S Jan24 0:00 tail -F /mnt/services/APP_THREE/var/log/celeryd.err
flume-ng 27945 0.0 0.0 3812 564 ? S Jan24 0:00 tail -F /mnt/services/APP_THREE/var/log/celerybeat.err
flume-ng 27951 0.0 0.0 3812 564 ? S Jan24 0:00 tail -F /mnt/services/MY_APP/var/log/celeryd.log
flume-ng 27967 0.0 0.0 3816 580 ? S Jan24 0:00 tail -F /mnt/services/APP_THREE/var/log/celeryd.log
flume-ng 27969 0.0 0.0 3820 528 ? S Jan24 0:00 tail -F /mnt/services/MY_APP/var/log/celerybeat.log
flume-ng 27970 0.0 0.0 3820 528 ? S Jan24 0:00 tail -F /mnt/services/APP_FOUR/var/log/celeryd.err
flume-ng 27974 0.0 0.0 3816 568 ? S Jan24 0:00 tail -F /mnt/services/APP_THREE/var/log/celerybeat.log
flume-ng 27977 0.0 0.0 3812 564 ? S Jan24 0:00 tail -F /mnt/services/MY_APP/var/log/celeryd.err
flume-ng 28050 0.0 0.0 3816 520 ? S Jan24 0:00 tail -F /mnt/services/APP_TWO/var/log/celeryd.log
508 9256 0.0 0.3 197348 29076 ? Sl Feb08 0:04 python2.7 /mnt/services/APP_THREE/bin/celery worker -B -Q APP_THREE --loglevel=INFO --autoreload -A APP_THREE.queue.tasks -E
508 9264 0.0 0.3 200584 27656 ? S Feb08 0:00 python2.7 /mnt/services/APP_THREE/bin/celery worker -B -Q APP_THREE --loglevel=INFO --autoreload -A APP_THREE.queue.tasks -E
508 9265 0.0 0.3 202092 28060 ? S Feb08 0:48 python2.7 /mnt/services/APP_THREE/bin/celery worker -B -Q APP_THREE --loglevel=INFO --autoreload -A APP_THREE.queue.tasks -E
508 9266 0.0 0.3 206420 29880 ? S Feb08 0:46 python2.7 /mnt/services/APP_THREE/bin/celery worker -B -Q APP_THREE --loglevel=INFO --autoreload -A APP_THREE.queue.tasks -E
APP_FOUR 14512 0.0 0.3 153144 23736 ? Sl 18:23 0:00 python2.7 /mnt/services/APP_FOUR/bin/celeryd --loglevel=INFO --autoreload -A APP_FOUR.queue.tasks -E -Q APP_FOUR
APP_FOUR 14545 0.0 0.2 136212 19868 ? S 18:23 0:00 python2.7 /mnt/services/APP_FOUR/bin/celeryd --loglevel=INFO --autoreload -A APP_FOUR.queue.tasks -E -Q APP_FOUR
APP_FOUR 14547 0.0 0.2 136232 19872 ? S 18:23 0:00 python2.7 /mnt/services/APP_FOUR/bin/celeryd --loglevel=INFO --autoreload -A APP_FOUR.queue.tasks -E -Q APP_FOUR
507 23894 14.6 0.3 154644 25168 ? Sl 19:29 2:08 [[email protected]:MainProcess] -active- (worker --beat --schedule=/mnt/services/MY_APP/var/lib/celerybeat --loglevel=INFO --autoreload --app=MY_APP.queue.tasks --events --queues=MY_APP)
507 23901 0.0 0.2 147852 22640 ? S 19:29 0:00 [celerybeat]
507 23902 6.1 0.3 143500 26312 ? S 19:29 0:53 [[email protected]:PoolWorker-2]
507 23903 6.1 0.3 143660 26452 ? S 19:29 0:54 [[email protected]:PoolWorker-3]
507 25859 0.0 0.0 6040 676 pts/0 S+ 19:43 0:00 grep celery