如何爲td-agent/fluentd指定自簽名的SSL CERT？

Question

我不斷收到以下錯誤：

2015-06-16 05:21:17 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2015-06-16 05:21:17 +0000 error_class="Faraday::SSLError" error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." plugin_id="object:127283c" 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `connect' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `initialize' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `new' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `socket' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:106:in `request_call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/mock.rb:47:in `request_call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/instrumentor.rb:22:in `request_call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:233:in `request' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/adapter/excon.rb:54:in `call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/rack_builder.rb:139:in `build_response' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/connection.rb:377:in `run_request' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:21:in `block in perform_request' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `call' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `perform_request' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in `perform_request' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:20:in `block in ping' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/utils.rb:189:in `__rescue_from_not_found' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:19:in `ping' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:64:in `client' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:171:in `rescue in send' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:169:in `send' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:163:in `write' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:325:in `write_chunk' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:304:in `pop' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:321:in `try_flush' 
    2015-06-16 05:21:17 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:140:in `run' 
2015-06-16 05:21:18 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2015-06-16 05:21:19 +0000 error_class="Faraday::SSLError" error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." plugin_id="object:127283c" 
    2015-06-16 05:21:18 +0000 [warn]: suppressed same stacktrace 
2015-06-16 05:21:20 +0000 [warn]: temporarily failed to flush the buffer. next_retry=2015-06-16 05:21:24 +0000 error_class="Faraday::SSLError" error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." plugin_id="object:127283c" 
    2015-06-16 05:21:20 +0000 [warn]: suppressed same stacktrace 
^C2015-06-16 05:21:22 +0000 [info]: shutting down fluentd 
2015-06-16 05:21:23 +0000 [warn]: before_shutdown failed error="Unable to verify certificate, please set `Excon.defaults[:ssl_ca_path] = path_to_certs`, `ENV['SSL_CERT_DIR'] = path_to_certs`, `Excon.defaults[:ssl_ca_file] = path_to_file`, `ENV['SSL_CERT_FILE'] = path_to_file`, `Excon.defaults[:ssl_verify_callback] = callback` (see OpenSSL::SSL::SSLContext#verify_callback), or `Excon.defaults[:ssl_verify_peer] = false` (less secure)." 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `connect' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/ssl_socket.rb:129:in `initialize' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `new' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:387:in `socket' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:106:in `request_call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/mock.rb:47:in `request_call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/instrumentor.rb:22:in `request_call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/excon-0.45.3/lib/excon/connection.rb:233:in `request' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/adapter/excon.rb:54:in `call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/rack_builder.rb:139:in `build_response' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.1/lib/faraday/connection.rb:377:in `run_request' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:21:in `block in perform_request' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `call' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:190:in `perform_request' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in `perform_request' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:20:in `block in ping' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/utils.rb:189:in `__rescue_from_not_found' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/ping.rb:19:in `ping' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:64:in `client' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:171:in `rescue in send' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:169:in `send' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-elasticsearch-0.9.0/lib/fluent/plugin/out_elasticsearch.rb:163:in `write' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:325:in `write_chunk' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/buffer.rb:304:in `pop' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/buf_memory.rb:93:in `block in before_shutdown' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/plugin/buf_memory.rb:89:in `before_shutdown' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:396:in `before_shutdown' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:160:in `block in run' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:159:in `synchronize' 
    2015-06-16 05:21:23 +0000 [warn]: /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.7/lib/fluent/output.rb:159:in `run' 
2015-06-16 05:21:23 +0000 [info]: process finished code=0 

我試過設置以下的環境變量：

• 設置SSL_CERT_FILE=/path/to/my/cert.crt
• 設置SSL_CERT_DIR=/path/to/my
• 設置SSL_CERT_DIR=/etc/ssl/certs
• 設置SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

我試過創建/home/td-agent/.bash_profile並更新/home/td-agent/.bashrc文件來導出類似的環境變量。當我運行td-agent服務時，我仍然遇到這個錯誤。

我試過在前面提到的env vars在命令行上運行td-agent服務。

我已經運行dpkg-reconfigure ca-certificates與我的自簽名證書適當的地方。 /usr/share/ca-certificates/<my cert dir>/my-self-signed-cert.crt

我試過運行沒有SSL_CERT_DIR或SSL_CERT_FILE環境變量集的td-agent。

注意：我能夠捲曲https://my.elasticsearch.myltd並得到200響應，所以我知道os（Ubuntu）拿起了我的證書。

至於版本，我運行Ubuntu 14.04 LTS，當我開始了TD-代理，下面是打印到日誌文件/var/log/td-agent/td-agent.log：找到答案在這裏

2015-06-16 05:07:05 +0000 [info]: reading config file path="/etc/td-agent/td-agent.conf" 
2015-06-16 05:07:05 +0000 [info]: starting fluentd-0.12.7 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-mixin-config-placeholders' version '0.3.0' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '0.9.0' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-mongo' version '0.7.8' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.4.1' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-s3' version '0.5.9' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-s3' version '0.5.7' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-td' version '0.10.26' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.0' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluent-plugin-webhdfs' version '0.4.1' 
2015-06-16 05:07:06 +0000 [info]: gem 'fluentd' version '0.12.7' 

Answer 1

：https://groups.google.com/forum/#!topic/fluentd/z-1vIsQ4kHU

原來，爲了設置td-agent服務的環境變量，您需要導出文件/etc/default/td-agent中的變量。

例如：

# This file is sourced by /bin/sh from /etc/init.d/td-agent 
# Options to pass to td-agent 
DAEMON_ARGS="" 
export SSL_CERT_DIR=/etc/ssl/certs 
export MYOTHERVAR=jazzjazzjazz