2013-10-28 40 views
2

我想使用gevent在python 2.7上使用brod寫kafka 0.7.2。如何在gevent中使用python kafka客戶端 - 是否有實際工作的庫文件?

這是我收到的錯誤消息。猜猜它是由於阻塞。布羅德支持龍捲風,但我使用gevent。

No handlers could be found for logger "brod.socket" 
Traceback (most recent call last): 
    File "/var/chef/cache/src/gevent/gevent/greenlet.py", line 328, in run 
    result = self._run(*self.args, **self.kwargs) 
    File "worker_server.py", line 204, in execute_kafka_pipe 
    kafka.produce(topic,payload) 
    File "/usr/local/lib/python2.7/dist-packages/brod-0.3.2-py2.7.egg/brod/base.py", line 287, in produce 
    return self._write(request, callback) 
    File "/usr/local/lib/python2.7/dist-packages/brod-0.3.2-py2.7.egg/brod/blocking.py", line 98, in _write 
    return self._write(data, callback, retries) 
    File "/usr/local/lib/python2.7/dist-packages/brod-0.3.2-py2.7.egg/brod/blocking.py", line 89, in _write 
    wrote_length += self._socket.send(data) 
    File "/var/chef/cache/src/gevent/gevent/socket.py", line 441, in send 
    self._wait(self._write_event) 
    File "/var/chef/cache/src/gevent/gevent/socket.py", line 292, in _wait 
    assert watcher.callback is None, 'This socket is already used by another greenlet: %r' % (watcher.callback,) 
AssertionError: This socket is already used by another greenlet: <bound method Waiter.switch of <gevent.hub.Waiter object at 0x1dece60>> 
<Greenlet at 0x1d4b9b0: execute_kafka_pipe('topic-spend', '{"enode": 1, "city": "Cairns", "dl": "en", "wnode)> failed with AssertionError 

我試圖使用gevent-kakfa,但依賴於gevent-zookeeper。

當試圖連接到飼養員我得到這個消息:

Traceback (most recent call last): 
    File "/home/ubuntu/workspace/rtbhui-devops/servers/worker_server.py", line 68, in <module> 
    framework = gevent_zookeeper.ZookeeperFramework('localhost:2181', 10) 
    File "/usr/local/lib/python2.7/dist-packages/gevent_zookeeper/framework.py", line 241, in __init__ 
    self.client = ZookeeperClient(hosts, timeout) 
    File "/usr/local/lib/python2.7/dist-packages/gevent_zookeeper/client.py", line 211, in __init__ 
    self._event = gevent.core.event(
AttributeError: 'module' object has no attribute 'core' 
Exception AttributeError: "'ZookeeperClient' object has no attribute '_event'" in <bound method ZookeeperClient.__del__ of <gevent_zookeeper.client.ZookeeperClient object at 0x274ded0>> ignored 

是不是有我,我可以寫使用GEVENT的作品的消息一個python的lib?

+0

我正在尋找相同的,並遇到這個https://github.com/duanhongyi/kafka。由於我自己沒有嘗試過,所以我在這裏注意它,而不是作爲答案。 – kkurian

+0

此外,https://github.com/mahendra/kafka-python/tree/gevent聲稱它有效但尚未向原始kafka-python b/c提交拉取請求,這取決於正在合併的另一個拉取請求第一。 – kkurian

+0

如果你發現一個與kafka 8和gevnet一起工作的lib ..請讓我知道,但會嘗試上面的鏈接 – Tampa

回答

1

嘗試Pykafka

雖然你應該能夠配置kafka- python通過傳入由gevent使用的selector

相關問題