芹菜：socket.error超時

我使用：芹菜：socket.error超時

celery==3.1.12 
kombu==3.0.19 
amqp==1.4.5

芹菜工人開始：

celery worker --without-gossip --without-heartbeat --without-mingle

芹菜setiings：

CELERY_ALWAYS_EAGER = False 
CELERY_EAGER_PROPAGATES_EXCEPTIONS = True 
BROKER_URL = 'amqp://guest:[email protected]:5672//' 
BROKER_CONNECTION_TIMEOUT = 30 
CELERY_RESULT_BACKEND = "amqp" 
CELERY_TASK_RESULT_EXPIRES = 18000 # 5 hours. 
CELERY_RESULT_PERSISTENT = True

有時候，我得到這個錯誤：

Traceback (most recent call last): 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 112, in get_response 
    response = wrapped_callback(request, *callback_args, **callback_kwargs) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/newrelic-2.18.1.15/newrelic/hooks/framework_django.py", line 492, in wrapper 
    return wrapped(*args, **kwargs) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/django_restapi/resource.py", line 143, in __call__ 
    return self.dispatch(request, self, *args, **kwargs) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/django_restapi/resource.py", line 76, in dispatch 
    return target.create(request, *args, **kwargs) 

    File "./kavyarnya/api/post_deploy.py", line 110, in create 
    process_commit.delay(project.id, commit, post_deploy_call.id) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/celery/app/task.py", line 358, in delay 
    return self.apply_async(args, kwargs) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/celery/app/task.py", line 474, in apply_async 
    **options) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/celery/app/amqp.py", line 249, in publish_task 
    **kwargs 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/messaging.py", line 164, in publish 
    routing_key, mandatory, immediate, exchange, declare) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/connection.py", line 470, in _ensured 
    interval_max) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/connection.py", line 396, in ensure_connection 
    interval_start, interval_step, interval_max, callback) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/utils/__init__.py", line 217, in retry_over_time 
    return fun(*args, **kwargs) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/connection.py", line 246, in connect 
    return self.connection 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/connection.py", line 761, in connection 
    self._connection = self._establish_connection() 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/connection.py", line 720, in _establish_connection 
    conn = self.transport.establish_connection() 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 115, in establish_connection 
    conn = self.Connection(**opts) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/amqp/connection.py", line 136, in __init__ 
    self.transport = create_transport(host, connect_timeout, ssl) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/amqp/transport.py", line 264, in create_transport 
    return TCPTransport(host, connect_timeout) 

    File "/usr/local/lib/staging/kava/.envf322d0e225b48a1bb90126f74d73812df3b92994/local/lib/python2.7/site-packages/amqp/transport.py", line 99, in __init__ 
    raise socket.error(last_err) 

error: timed out

RabbitMq有瑕疵設置。 https://www.rabbitmq.com/configure.html

/etc/default/rabbitmq-server : `ulimit -n 16384`

兔日誌：

=WARNING REPORT==== 19-Mar-2016::00:02:17 === 
closing AMQP connection <0.30881.4> (127.0.0.1:26762 -> 127.0.0.1:5672): 
connection_closed_abruptly

套接字的最大數量：

# sysctl -a | grep somax 
net.core.somaxconn = 32768

我試圖通過以下解決此問題方法：

Set BROKER_HEARTBEAT = 10 - 我有更糟糕的情況下
設置BROKER_POOL_LIMIT =無 - 什麼都沒有改變對當前狀態
設置BOKER_CONNECTION_TIMEOUT至10，20，30 - 更大的超時值帶來的誤差較小

我認爲增加超時值只是隱瞞真實問題。這個問題的原因是什麼，我該如何解決？謝謝

來源

2016-03-24 Begun-A

您試圖處理的請求的比率是多少？

你能分享一下，在運行RabbitMQ的計算機上CPU利用率，內存消耗，磁盤IO狀態會發生什麼情況，你會看到超時？

看起來像兔子不能足夠快地處理連接。

由於盲目的猜測，我建議增加二郎山VM I/O線程池的大小（請參閱「二郎山VM I/O線程池」一節在這裏https://www.rabbitmq.com/networking.html）

也許與兔子的「連接積壓」打了一下（看到相同的鏈接）。

總有一種可能性，您沒有足夠的資源來滿足您要求的數量，因此您可能需要擴展Rabbit安裝。

來源

2016-03-25 11:08:19

芹菜：socket.error超時

回答

相關問題