發現差異，芹菜任務隨機失敗，沒有錯誤

我的一些遠程Celery任務似乎從未讓我的經紀人（RabbitMQ）。這appears隨機發生。在我的日誌中有NO錯誤，並且他們從未將其發送給工作人員或失敗。花/兔從不報告任務失敗。發現差異，芹菜任務隨機失敗，沒有錯誤

我用tcpflow -p -c -i eth0 port 5672來監視發送任務(client)的API的流量。

當API成功發送任務傳出的流量是記錄如下：

（刪除敏感數據）

192.018.000.002.42738-052.048.150.171.05672: AMQP 
052.048.150.171.05672-192.018.000.002.42738: 

capabilitiesFpublisher_confirmstexchange_exchange_bindingst 
basic.nacktconsumer_cancel_notifytconnection.blockedtconsumer_prioritiestau[email protected]d8b85eb5ab91copyrightS.Copyright (C) 2007-2015 Pivotal Software, Inc.informationS5Licensed under the MPL. See http://www.rabbitmq.com/platformS 
Erlang/OTPproductSRabbitMQversionS3.6.0PLAIN AMQPLAINen_US 
192.018.000.002.42738-052.048.150.171.05672: 
nproductSpy-amqpproduct_versionS1.4.9capabilitiesF.connection.blockedtconsumer_cancel_notifytAMQPLAIN1LOGINSusernamePASSWORDSxxxxxxen_US 
052.048.150.171.05672-192.018.000.002.42738: 
< 
192.018.000.002.42738-052.048.150.171.05672: 

192.018.000.002.42738-052.048.150.171.05672: 
(/ 
052.048.150.171.05672-192.018.000.002.42738: 
) 
192.018.000.002.42738-052.048.150.171.05672: 

052.048.150.171.05672-192.018.000.002.42738: 
192.018.000.002.42738-052.048.150.171.05672: $(
estimate_geometrydirect 
052.048.150.171.05672-192.018.000.002.42738: (
192.018.000.002.42738-052.048.150.171.05672: 2 
estimate_geometry 
052.048.150.171.05672-192.018.000.002.42738: 2estimate_geometry 
192.018.000.002.42738-052.048.150.171.05672: G2estimate_geometryestimate_geometrytasks.estimate_geometry 
052.048.150.171.05672-192.018.000.002.42738: 2 
192.018.000.002.42738-052.048.150.171.05672: 1<(estimate_geometrytasks.estimate_geometry 
192.018.000.002.42738-052.048.150.171.05672: <application/x-python-serializebinary$021e5308-e6ac-43eb-9a06-8473ba386802$bedeb08f-9614-38b1-9b60-9eded43c3c71 
192.018.000.002.42738-052.048.150.171.05672: }q(UexpiresqNUutcqUargsq]qCaUchordqNUcallbacksqNUerrbacksqNUtasksetqNUidq 
Utasks.estimate_geometryqUtimelimitqNNUetaqNUkwargsq}qu. 
192.018.000.002.42738-052.048.150.171.05672: (
segment_imagedirect 
052.048.150.171.05672-192.018.000.002.42738: (
192.018.000.002.42738-052.048.150.171.05672: 2 
segment_image 
segment_image71.05672-192.018.000.002.42738: 2 
segment_imagetasks.segment_image0.171.05672: ;2 
052.048.150.171.05672-192.018.000.002.42738: 2 
segment_imagetasks.segment_image0.171.05672:)<(
192.018.000.002.42738-052.048.150.171.05672: <application/x-python-serializebinary$45280975-9611-41e1-bf99-388cdf1b7064$bedeb08f-9614-38b1-9b60-9eded43c3c71 
192.018.000.002.42738-052.048.150.171.05672: }q(UexpiresqNUutcqUargsq]qCaUchordqNUcallbacksqNUerrbacksqNUtasksetqNUidq 
Utasks.segment_imageqUtimelimitqNNUetaqNUkwargsq}qu.skq

這是一個任務從未使它到代理誰能找出差異並告訴我最新的錯誤？

192.018.000.002.35908-052.017.119.221.05672: 1<(estimate_geometrytasks.estimate_geometry 
192.018.000.002.35908-052.017.119.221.05672: <application/x-python-serializebinary$206c1ae0-43d0-4031-bac6-92d2df92b13c$f4c7420c-b9c2-3525-bd5a-d5955f884f43 
192.018.000.002.35908-052.017.119.221.05672: }q(UexpiresqNUutcqUargsq]qCaUchordqNUcallbacksqNUerrbacksqNUtasksetqNUidq 
Utasks.estimate_geometryqUtimelimitqNNUetaqNUkwargsq}qu. 
segment_imagetasks.segment_image9.221.05672:)<(
192.018.000.002.35908-052.017.119.221.05672: <application/x-python-serializebinary$ce0da18a-6534-42d0-9919-cd2e85c8d5e9$f4c7420c-b9c2-3525-bd5a-d5955f884f43 
192.018.000.002.35908-052.017.119.221.05672: }q(UexpiresqNUutcqUargsq]qCaUchordqNUcallbacksqNUerrbacksqNUtasksetqNUidq 
Utasks.segment_imageqUtimelimitqNNUetaqNUkwargsq}qu.skq

附加信息：

celery_app = Celery('tasks') 
celery_app.config_from_object('django.conf:settings') 
celery_app.autodiscover_tasks(lambda: settings.INSTALLED_APPS) 

celery_app.send_task('tasks.estimate_geometry', args=[instance.id], kwargs={})

設置：

import os 
from kombu import Exchange, Queue 

BROKER_URL = 'amqp://xxxx:[email protected]:5672//' 

CELERY_RESULT_BACKEND = "cache" 
CELERY_CACHE_BACKEND = 'memcached://xxxxxx:11211' 

CELERY_DEFAULT_QUEUE = 'default' 
CELERY_DEFAULT_EXCHANGE_TYPE = 'topic' 
CELERY_DEFAULT_ROUTING_KEY = 'default' 
CELERY_QUEUES = (
    Queue('default', Exchange('default'), routing_key='default'), 
    Queue('estimate_geometry', Exchange('estimate_geometry'), routing_key='tasks.estimate_geometry'), 
    Queue('segment_image', Exchange('segment_image'), routing_key='tasks.segment_image'), 
    Queue('geometry_feature', Exchange('geometry_feature'), routing_key='tasks.geometry_feature'), 
) 
CELERY_ROUTES = { 
    'tasks.estimate_geometry': { 
     'queue': 'estimate_geometry', 
     'routing_key': 'tasks.estimate_geometry', 
    }, 
    'tasks.segment_image': { 
     'queue': 'segment_image', 
     'routing_key': 'tasks.segment_image', 
    }, 
    'tasks.geometry_feature': { 
     'queue': 'geometry_feature', 
     'routing_key': 'tasks.geometry_feature', 
    }, 
} 


BROKER_HEARTBEAT = 10

來源

2016-02-12 Glyn Jackson

你怎麼知道它從來沒有讓它給經紀人？ – scytale

它看起來就像你在調試問題採取了錯誤的道路。 I see line：

segment_imagetasks.segment_image0.171.05672:)<("

在兩個日誌中。

如果是從您的代碼打印輸出，那麼它是工作，但不正確。所以麻煩不在傳輸層。

在第一種情況下，我可以看到行：

192.018.000.002.42738-052.048.150.171.05672: (
segment_imagedirect 
052.048.150.171.05672-192.018.000.002.42738: (
192.018.000.002.42738-052.048.150.171.05672: 2 
segment_image 
segment_image71.05672-192.018.000.002.42738: 2 
segment_imagetasks.segment_image0.171.05672: ;2 
052.048.150.171.05672-192.018.000.002.42738: 2

所以，有時它運作良好。

你應該使用實例id來手動運行你的代碼（tasks.estimate_geometry）。

如果是服務器日誌：可能是某些未在服務器上編譯的c-libraries，並且您的代碼在本地工作，但在服務器上不起作用。所以，檢查圖像格式。

來源

2016-02-18 16:51:52

發現差異，芹菜任務隨機失敗，沒有錯誤

回答

相關問題