2017-09-03 59 views
2

我一直在試圖連接使用卡夫卡卡夫卡連接api.Kafka版本elasticsearch是0.11.0.0.These是我遵循的步驟:

1.Buiding Elasticsearch連接器:

https://github.com/confluentinc/kafka-connect-elasticsearch.git

2.Build連接器

$ cd kafka-connect-elasticsearch 
$ mvn clean package 

3.最後運行腳本:

$ bin/connect-standalone.sh config/connect-standalone.properties config/elasticsearch-connect.properties 

它拋出以下異常:

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.collect.Sets$SetView.iterator()Lcom/google/common/collect/UnmodifiableIterator; 
    at org.reflections.Reflections.expandSuperTypes(Reflections.java:380) 
    at org.reflections.Reflections.<init>(Reflections.java:126) 
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:221) 
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:198) 
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:159) 
    at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47) 
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:68) 

不明白什麼地方出了錯。

+0

也許只是嘗試安裝包括0.11和Elasticsearch連接器的Confluent Open Source發行版,並按照快速入門http://docs.confluent.io/current/connect/connect-elasticsearch/docs/elasticsearch_connector。 html –

+0

我已經完成了整個代碼而沒有使用Confluent,現在切換它並不是我正在計劃的內容。是不是有解決方案? –

+0

這只是一個建議,可以幫助您確定問題是您的設置,構建環境還是Confluent ES連接器本身。 –

回答

1

這似乎描述的答案您的問題https://github.com/confluentinc/kafka-connect-elasticsearch/issues/104

It's a little confusing, but after you build the connector there are a number of things in the target directory. The kafka-connect-elasticsearch-.jar is only the JAR file with the connector code, but that doesn't include all the libraries. One of those directories in the target directory, namely target/kafka-connect-elasticsearch-*-development/share/java/kafka-connect-elasticsearch/, does contain all the libraries. Add this directory to the Kafka Connect worker's classpath, or copy all of those JAR files into a directory that is already on the classpath.

1

從經驗來看,這種錯誤意味着你在你的類路徑帶來的guava舊版本更早。連接工作人員要求guava >= 20org.reflections才能正常工作。

kafka-connect-elasticsearch或與之guava 18.0以上的老年人帶來的將啓動禁止工人任何其他連接器。此錯誤消息表示在類路徑中遇到了較早的guava jar。

兩個解決方案:

  1. 的確,作爲Hans Jespersen提到,使用類加載隔離通過設置plugin.path在連接勞動者的配置,將允許連接器工作原樣不與連接框架干擾。
  2. 如果將連接器添加到CLASSPATH是您唯一的選擇,請確保它是在Kafka Connect的依賴關係後添加的,以便最近的guava將被選中。