導致卡夫卡超時的Zookeeper f-sync問題

我們正在嘗試在16核心的虛擬機上執行單節點安裝& 16 GB RAM用於Zookeeper & Kafka。 Zookeeper顯示f-sync問題，在設定的6000毫秒超時時間後，kafka發出超時。導致卡夫卡超時的Zookeeper f-sync問題

附加信息：動物園管理員版本：3.4.6-1569965 卡夫卡版本：2.11-0.9.0.1

下面的條目都在那裏ZK日誌：

[2017-03-06 16:37:18,013] INFO Processed session termination for sessionid: 0x15aa58d60530000 (org.apache.zookeeper.server.PrepRequestProcessor) 
[2017-03-06 16:37:45,308] WARN fsync-ing the write ahead log in SyncThread:0 took 35773ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide (org.apache.zookeeper.server.persistence.FileTxnLog) 
[2017-03-06 16:37:45,334] INFO Established session 0x15aa58d60530000 with negotiated timeout 6000 for client /127.0.0.1:44860 (org.apache.zookeeper.server.ZooKeeperServer) 
[2017-03-06 16:37:45,340] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn) 
EndOfStreamException: Unable to read additional data from client sessionid 0x15aa58d60530000, likely client has closed socket 
     at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) 
     at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) 
     at java.lang.Thread.run(Thread.java:745) 
[2017-03-06 16:37:45,347] INFO Closed socket connection for client /127.0.0.1:44860 which had sessionid 0x15aa58d60530000 (org.apache.zookeeper.server.NIOServerCnxn) 
[2017-03-06 16:37:50,360] WARN fsync-ing the write ahead log in SyncThread:0 took 5009ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide (org.apache.zookeeper.server.persistence.FileTxnLog)

卡夫卡有超時錯誤ZK ：

[2017-03-06 16:10:11,115] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) 
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000 
       at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223) 
       at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:155) 
       at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:129) 
       at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89) 
       at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71) 
       at kafka.server.KafkaServer.initZk(KafkaServer.scala:278) 
       at kafka.server.KafkaServer.startup(KafkaServer.scala:168) 
       at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37) 
       at kafka.Kafka$.main(Kafka.scala:67) 
       at kafka.Kafka.main(Kafka.scala) 
[2017-03-06 16:10:11,120] INFO shutting down (kafka.server.KafkaServer) 
[2017-03-06 16:10:11,128] INFO shut down completed (kafka.server.KafkaServer) 
[2017-03-06 16:10:11,129] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable) 
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000 
       at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223) 
       at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:155) 
       at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:129) 
       at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89) 
       at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71) 
       at kafka.server.KafkaServer.initZk(KafkaServer.scala:278) 
       at kafka.server.KafkaServer.startup(KafkaServer.scala:168) 
       at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37) 
       at kafka.Kafka$.main(Kafka.scala:67) 
       at kafka.Kafka.main(Kafka.scala) 
[2017-03-06 16:10:11,140] INFO shutting down (kafka.server.KafkaServer)

如果這是已知的問題與任何可用的解決方法請分享任何鏈接/論壇，我可以跟進它。

感謝

來源

2017-03-08 Anshul Malik

檢查飼養員正在運行

你給在配置正確的端口

供消費者使用的動物園管理員端口（2181默認）和生產者使用卡夫卡端口（6667默認）

來源

2017-03-08 19:58:00

導致卡夫卡超時的Zookeeper f-sync問題

回答

相關問題