fluentd無法/顯着連接到elasticsearch當ES安裝在不同的端口

在我的碼頭環境中有彈性搜索（ES-1）暴露於端口9200，這是主要ES維護DB數據，所以爲了日誌記錄我們使用fluentd-elasticsearch-kibna堆棧，並在端口9700（ES-2）上運行新的彈性搜索。fluentd無法/顯着連接到elasticsearch當ES安裝在不同的端口

我無法fluentd連接到這個新的彈性搜索（ES-2）端口9700。讓我知道如何連接fluentd到上課比其他端口「9200」

下面

是使用泊塢窗的配置細節對於FEK

fluentd.config : 
    <source> 
    @type forward 
    port 24224 
    bind 0.0.0.0 
</source> 

<match *.**> 
    @type copy 
    <store> 
    @type elasticsearch 
    host elasticsearchkibana 
    port 9200 
    logstash_format true 
    logstash_prefix fluentd 
    logstash_dateformat %Y%m%d 
    include_tag_key true 
    type_name access_log 
    tag_key @log_name 
    flush_interval 1s 
    </store> 
    <store> 
    @type stdout 
    </store> 
</match> 


docker-compose.yml : 

    version: '2' 
services: 
    web: 
    image: httpd 
    ports: 
     - "80:80" 
    links: 
     - fluentd 
    logging: 
     driver: "fluentd" 
     options: 
     fluentd-address: localhost:24224 
     tag: httpd.access 

    elasticsearchkibana: 
    image: elasticsearch 
    environment: 
     - "ES_JAVA_OPTS=-Xms512m -Xmx512m" 
    expose: 
     - 9800 
    ports: 
     - "9700:9300" 
     - "9800:9200" 

    fluentd: 
    build: ./fluentd 
    volumes: 
     - ./fluentd/conf:/fluentd/etc 
    links: 
     - "elasticsearchkibana" 
    ports: 
     - "24224:24224" 
     - "24224:24224/udp" 

    kibana: 
    image: kibana 
    environment: 
     - ELASTICSEARCH_URL=http://elasticsearchkibana:9200 
#  - "ES_HOST=192.168.241.82" 
#  - "ES_PORT=9700" 
    ports: 
     - "5601:5601"

下面是錯誤消息我收到當我使用上面的配置

fluentd_1    | 2017-09-25 15:16:36 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:37 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:37 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:37 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:39 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:37 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:37 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:39 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:39 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:39 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:43 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:39 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:39 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:43 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:43 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:43 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:50 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:43 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:43 +0000 fluent.warn: {"next_retry":"2017-09-25 15:16:50 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:16:50 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:16:50 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:04 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:16:50 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:16:50 +0000 fluent.warn: {"next_retry":"2017-09-25 15:17:04 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:04 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:17:04 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:39 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:17:04 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:17:04 +0000 fluent.warn: {"next_retry":"2017-09-25 15:17:39 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:17:39 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:17:39 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:18:46 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:17:39 +0000 fluent.warn: {"next_retry":"2017-09-25 15:18:46 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:18:46 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
fluentd_1    | 2017-09-25 15:17:39 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:18:46 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2017-09-25 15:20:49 +0000 error_class="Fluent::ElasticsearchOutput::ConnectionFailure" error="Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!" plugin_id="object:2af57e68ced4" 
fluentd_1    | 2017-09-25 15:18:46 +0000 [warn]: suppressed same stacktrace 
fluentd_1    | 2017-09-25 15:18:46 +0000 fluent.warn: {"next_retry":"2017-09-25 15:20:49 +0000","error_class":"Fluent::ElasticsearchOutput::ConnectionFailure","error":"Can not reach Elasticsearch cluster ({:host=>\"localhost\", :port=>9700, :scheme=>\"http\"})!","plugin_id":"object:2af57e68ced4","message":"temporarily failed to flush the buffer. next_retry=2017-09-25 15:20:49 +0000 error_class=\"Fluent::ElasticsearchOutput::ConnectionFailure\" error=\"Can not reach Elasticsearch cluster ({:host=>\\\"localhost\\\", :port=>9700, :scheme=>\\\"http\\\"})!\" plugin_id=\"object:2af57e68ced4\""} 
[email protected]:/home/tlirlogin/docker-compose-efk-master2# cd fluentd/conf/

感謝

來源

2017-09-25 k.explorer

您的fluentd運行在容器網絡內而不是主機。所以即使您在主機上映射到9700，端口仍然需要9200。因爲在這種情況下連接取決於容器端口 –

@tarun Laiwani 更改時仍然出錯 fluentd_1 fluent.warn：error_class「：」Fluent :: ElasticsearchOutput :: ConnectionFailure「，」error「：」無法到達Elasticsearch集羣（{：host => \「elasticsearchkibana \」，：port => 9200，：scheme => \「http \」}）！「，」plugin_id「：」object：2ac5be462aa4「，」message「：」暫時失敗刷新緩衝區。 next_retry = 2017-09-26 05:14:24 +0000 error_class = \「Fluent :: ElasticsearchOutput :: ConnectionFailure \」error = \「無法到達Elasticsearch集羣 –

您能分享fluentd dockerfile嗎？所以我可以給你寫作文件一射 –

你需要改變你的fluentd配置如下

fluentd.config : 
<source> 
    @type forward 
    port 24224 
    bind 0.0.0.0 
</source> 

<match *.**> 
    @type copy 
    <store> 
    @type elasticsearch 
    host elasticsearchkibana 
    port 9200 
    logstash_format true 
    logstash_prefix fluentd 
    logstash_dateformat %Y%m%d 
    include_tag_key true 
    type_name access_log 
    tag_key @log_name 
    flush_interval 1s 
    </store> 
    <store> 
    @type stdout 
    </store> 
</match>

所以基本上添加host elasticsearchkibana並更改爲port 9200。這是因爲我們應該關注容器端口而不是主機端口

來源

2017-09-25 17:59:46

fluentd無法/顯着連接到elasticsearch當ES安裝在不同的端口

回答

相關問題