如何檢查Kafka Consumer中file.txt的輸出？

Question

我正在嘗試將Flume與Kafka集成，我想將文件從本地計算機傳遞給Flume，然後傳遞給Kafka。我想在Kafka Consumer中查看我的文件內容。

這裏是我的水槽的配置文件：

# example.conf: A single-node Flume configuration 

# Name the components on this agent 
a1.sources = r1 
a1.sinks = k1 
a1.channels = c1 

# Describe/configure the source 
a1.sources.r1.type = exec 
a1.sources.r1.command = cat /Users/myname/Downloads/file.txt 

# Describe the sink 
#a1.sinks.k1.type = logger 
    a1.sinks.k1.type   = org.apache.flume.sink.kafka.KafkaSink 
    a1.sinks.k1.topic = k1 
    a1.sinks.k1.brokerList = kafkagames-1:9092,kafkagames-2:9092 
    a1.sinks.sink1.batchSize = 20 

    # Use a channel which buffers events in memory 
    a1.channels.c1.type = memory 
    a1.channels.c1.capacity = 10000 
    a1.channels.c1.transactionCapacity = 10000 

    # Bind the source and sink to the channel 
    a1.sources.r1.channels = c1 
    a1.sinks.k1.channel = c1 

我不知道如何從水槽開始卡夫卡看file.txt的在卡夫卡消費者的內容。請指教。謝謝。

Answer 1

由於Kafka是一個消息代理「服務」，您必須在與您的Flume生產者生成消息之前運行它。在生產者產生（放入）一個消息給一個Kafka主題（一種緩衝區）之後，你可以使用一個Kafka消費者來消費（獲取）該消息。如何開始卡夫卡服務： http://kafka.apache.org/documentation.html#quickstart