消費kafka話題使用logstash elasticSearch

Question

我開始消費從kafka消息logstash，我想發送完整的主題elasticSearch，但我沒有得到任何消息logstash，我可以看到在kafka消息來了，但從卡夫卡方面我沒有看到任何東西，什麼是正確的方式來配置它？

input { 
    kafka { 
    zk_connect => "localhost:2181" 
    topic_id => "event" 
    } 
} 

output{ 
    stdout{ 
    codec => rubydebug 
    } 
    elasticsearch{ 
    index => "event-%{+YYYY.MM.dd}" 
    hosts => ["localhost:9201"] 
    codec => json 
    } 
} 

curl localhost:9201 
{ 
    "name" : "Flex", 
    "cluster_name" : "elasticsearch", 
    "version" : { 
    "number" : "2.3.4", 
    "build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f", 
    "build_timestamp" : "2016-06-30T11:24:31Z", 
    "build_snapshot" : false, 
    "lucene_version" : "5.5.0" 
    }, 
    "tagline" : "You Know, for Search" 
} 

命令：

/kafka-console-consumer.sh --zookeeper localhost:2181 --topic event 

產生的結果不時。

Answer 1

嘗試像這樣與auto_offset_reset和reset_beginning：

kafka { 
    topic_id => "event" 
    zk_connect => "localhost:2181" 
    group_id => "event-group" 
    auto_offset_reset => "smallest" 
    reset_beginning => true 
    consumer_threads => 1 
    }