不能將Celery服務器連接到本地主機上的RabbitMQ

Question

我正在使用Celery和RabbitMQ作爲消息隊列，其中每個封裝在它自己的Docker映像中。當它們在Docker中使用--link參數連接時，一切正常。我已經有這個設置工作了一段時間了。我想將它們分開，以便它們在不同的主機上運行，​​所以我不能再使用--link參數。當我嘗試使用AMQP進行連接並且不明白原因時，我收到了gaierror: [Errno -2] Name or service not known。

服務器只需使用rabbitmq容器上DockerHub：

docker run --rm --name=qrabbit -p 5672:5672 rabbitmq 

我可以telnet到這個成功：

$ telnet 192.168.99.100 5672 
Trying 192.168.99.100... 
Connected to 192.168.99.100. 
Escape character is '^]'. 
abc 
^D 
AMQP Connection closed by foreign host. 
$ 

...所以我知道該服務器正在運行。

我的客戶是這樣的：

import os 

from logging import getLogger, StreamHandler, DEBUG 
from serverlib import QueueServer, CeleryMonitor 
from celery import Celery 
from argparse import ArgumentParser 

log = getLogger('server') 
log.addHandler(StreamHandler()) 
log.setLevel(DEBUG) 

broker_service_host = os.environ.get('MESSAGE_QUEUE_SERVICE_SERVICE_HOST') 

broker = 'amqp://{0}'.format(broker_service_host) 
host = '' 
port = 8000 
retry = 5 

if __name__ == '__main__': 
    log.info('connecting to {0}, {1}:{2}, retry={3}'.format(broker, host, port, retry)) 
    app = Celery(broker=broker) 
    monitor = CeleryMonitor(app, retry=retry) 
    server = QueueServer((host, port), app) 
    monitor.start() 
    try: 
     log.info('listening on {0}:{1}'.format(host, port)) 
     server.serve_forever() 
    except KeyboardInterrupt: 
     log.info('shutdown requested') 
    except BaseException as e: 
     log.error(e) 
    finally: 
     monitor.shutdown() 

我有點某些外部模塊（QueueServer和CeleryMonitor）不是問題的一部分，因爲它運行正常，當我做到以下幾點：

$ docker run --rm --name=qmaster -e "MESSAGE_QUEUE_SERVICE_SERVICE_HOST=localhost" --link qrabbit:rabbit -p 80:8000 render-task-master 
connecting to amqp://localhost, :8000, retry=5 
listening on :8000 
^Cshutdown requested 
$ 

......但如果我做了以下（不--link參數）：

$ docker run --rm --name=qmaster -e "MESSAGE_QUEUE_SERVICE_SERVICE_HOST=localhost" -p 80:8000 render-task-master 
connecting to amqp://localhost, :8000, retry=5 
listening on :8000 
Exception in thread Thread-1: 
Traceback (most recent call last): 
    File "/usr/local/lib/python2.7/threading.py", line 810, in __bootstrap_inner 
    self.run() 
    File "/home/celery/serverlib/celerymonitor.py", line 68, in run 
    '*': self.__state.event 
    File "/usr/local/lib/python2.7/site-packages/celery/events/__init__.py", line 287, in __init__ 
    self.channel = maybe_channel(channel) 
    File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 1054, in maybe_channel 
    return channel.default_channel 
    File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 756, in default_channel 
    self.connection 
    File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 741, in connection 
    self._connection = self._establish_connection() 
    File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 696, in _establish_connection 
    conn = self.transport.establish_connection() 
    File "/usr/local/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 116, in establish_connection 
    conn = self.Connection(**opts) 
    File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 165, in __init__ 
    self.transport = self.Transport(host, connect_timeout, ssl) 
    File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 186, in Transport 
    return create_transport(host, connect_timeout, ssl) 
    File "/usr/local/lib/python2.7/site-packages/amqp/transport.py", line 299, in create_transport 
    return TCPTransport(host, connect_timeout) 
    File "/usr/local/lib/python2.7/site-packages/amqp/transport.py", line 75, in __init__ 
    socket.SOCK_STREAM, SOL_TCP): 
gaierror: [Errno -2] Name or service not known 

^Cshutdown requested 
$ 

使用和不使用可能導致此錯誤的參數--link有什麼區別？

UPDATE：

我已經收窄到一個錯誤的監視器類，我創建：

recv = self.app.events.Receiver(connection, handlers={ 
    'task-received': self.registerTask, 
    'task-failed': self.retryTask, 
    'task-succeeded': self.deregisterTask, 
    # should process all events to have state up to date 
    '*': self.__state.event 
}) 

如果這就是所謂的，它位於幾秒鐘（超時？）然後拋出一個異常。任何想法爲什麼這不喜歡指定爲amqp://localhost的amqp URL，但是當我使用--link參數時，一切正常？

這裏是整個方法調用中，額外的背景：

def run(self): 
    log.info('run') 
    self.__state = self.app.events.State() 
    with self.app.connection() as connection: 
     log.info('got a connection') 
     recv = self.app.events.Receiver(connection, handlers={ 
      'task-received': self.registerTask, 
      'task-failed': self.retryTask, 
      'task-succeeded': self.deregisterTask, 
      # should process all events to have state up to date 
      '*': self.__state.event 
     }) 
     log.info('received receiver') 
     # Capture until shutdown requested 
     while not self.__shutdown: 
      log.info('main run loop') 
      try: 
       recv.capture(limit=None, timeout=1, wakeup=True) 
      except timeout: 
       # timeout exception is fired when nothing occurs 
       # during timeout. Just ignore it. 
       pass 

Answer 1

我發現這個問題：我在其中容器運行多克爾環境設置CELERY_BROKER_URL，這是造成後端嘗試連接到不存在的主機名。一旦我取消設置變量，一切都會在我的環境中正確連接。

$ docker inspect server 
<... removed ...> 
     "Env": [ 
     "MESSAGE_QUEUE_SERVICE_SERVICE_HOST=192.168.99.100", 
     "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
     "LANG=C.UTF-8", 
     "PYTHON_VERSION=2.7.10", 
     "PYTHON_PIP_VERSION=7.1.2", 
     "CELERY_VERSION=3.1.18", 
     "CELERY_BROKER_URL=amqp://guest@rabbit" 
    ], 
<... removed ...>