將Apache Kafka連接到數據庫

我想用PostgreSQL，Apache Kafka（發送數據）和Apache Spark Streaming（處理數據）創建流/大數據應用程序。將Apache Kafka連接到數據庫

繼此document後，我首先嚐試將Kafka連接到Sqlite數據庫。但是當我用命令運行連接器時

$ ./bin/connect-standalone ./etc/schema-registry/connect-avro-standalone.properties ./etc/kafka-connect-jdbc/source-quickstart-sqlite.properties

在進程關閉之前，我得到了下面的錯誤。我將正確的路徑傳遞給我的數據庫，我驗證了。

如何連接和工作我的數據庫之間的連接& Kafka？

2017-09-11 12:00:17,305] INFO jetty-9.2.15.v20160210 (org.eclipse.jetty.server.Server:327) 
Sep 11, 2017 12:00:17 PM org.glassfish.jersey.internal.Errors logErrors 
WARNING: The following warnings have been detected: WARNING: The (sub)resource method createConnector in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation. 
WARNING: The (sub)resource method listConnectors in org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource contains empty path annotation. 
WARNING: The (sub)resource method listConnectorPlugins in org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource contains empty path annotation. 
WARNING: The (sub)resource method serverInfo in org.apache.kafka.connect.runtime.rest.resources.RootResource contains empty path annotation. 

[2017-09-11 12:00:17,618] INFO Started [email protected]{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744) 
[2017-09-11 12:00:17,623] WARN FAILED [email protected]{HTTP/1.1}{0.0.0.0:8083}: java.net.BindException: Address already in use (org.eclipse.jetty.util.component.AbstractLifeCycle:212) 
java.net.BindException: Address already in use 
    at sun.nio.ch.Net.bind0(Native Method) 
    at sun.nio.ch.Net.bind(Net.java:433) 
    at sun.nio.ch.Net.bind(Net.java:425) 
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) 
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) 
    at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321) 
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) 
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) 
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
    at org.eclipse.jetty.server.Server.doStart(Server.java:366) 
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
    at org.apache.kafka.connect.runtime.rest.RestServer.start(RestServer.java:145) 
    at org.apache.kafka.connect.runtime.Connect.start(Connect.java:53) 
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:82) 
[2017-09-11 12:00:17,624] WARN FAILED [email protected]: java.net.BindException: Address already in use (org.eclipse.jetty.util.component.AbstractLifeCycle:212) 
java.net.BindException: Address already in use 
    at sun.nio.ch.Net.bind0(Native Method) 
    at sun.nio.ch.Net.bind(Net.java:433) 
    at sun.nio.ch.Net.bind(Net.java:425) 
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) 
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) 
    at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321) 
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) 
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) 
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
    at org.eclipse.jetty.server.Server.doStart(Server.java:366) 
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
    at org.apache.kafka.connect.runtime.rest.RestServer.start(RestServer.java:145) 
    at org.apache.kafka.connect.runtime.Connect.start(Connect.java:53) 
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:82) 
[2017-09-11 12:00:17,624] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:100) 
org.apache.kafka.connect.errors.ConnectException: Unable to start REST server 
    at org.apache.kafka.connect.runtime.rest.RestServer.start(RestServer.java:147) 
    at org.apache.kafka.connect.runtime.Connect.start(Connect.java:53) 
    at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:82) 
Caused by: java.net.BindException: Address already in use 
    at sun.nio.ch.Net.bind0(Native Method) 
    at sun.nio.ch.Net.bind(Net.java:433) 
    at sun.nio.ch.Net.bind(Net.java:425) 
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) 
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) 
    at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321) 
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) 
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) 
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
    at org.eclipse.jetty.server.Server.doStart(Server.java:366) 
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) 
    at org.apache.kafka.connect.runtime.rest.RestServer.start(RestServer.java:145) 
    ... 2 more

來源

2017-09-11 Baerrow

WARN FAILED [email protected]{HTTP/1.1}{0.0.0.0:8083}: java.net.BindException: Address already in use (org.eclipse.jetty.util.component.AbstractLifeCycle:212) 
java.net.BindException: Address already in use

這意味着別的東西是否使用Confluent Platform已在使用端口8083.

？這是開始使用Apache Kafka的好方法，併爲您提供了一個簡單的方法來調出堆棧（confluent start）以及配置Kafka Connect。此博客系列展示了它在行動：

順便說一句，你可能想看看Kafka Streams也一樣，內卡夫卡自己做流處理的方式。

來源

2017-09-12 06:42:08

我正在使用融合平臺，是的，我正在關注文檔。我只想在使用postgres數據庫進行操作之前嘗試使用sqlite數據庫。 – Baerrow

啊，這是有道理的。 '匯合開始'也將開始連接。因此，您可以使用'confluent load'來添加您擁有的配置文件，或者如果您想要按原樣使用快速啓動，請首先運行'confluent stop connect'，它將釋放端口8083. –

但是，連接對於數據庫連接。還是我真的不明白？ – Baerrow

將Apache Kafka連接到數據庫

回答

相關問題