正如標題所述,Logstash一旦啓動就會崩潰。
我正在通過Docker運行它。Logstash一開始崩潰
這是我的輸出:
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.470000+0000", :message=>"Reading config file", :config_file=>"/etc/logstash/conf.d/logstash.conf", :level=>:debug, :file=>"logstash/config/loader.rb", :line=>"69", :method=>"local_config"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.570000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"86", :method=>"lookup"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.584000+0000", :message=>"starting agent", :level=>:info, :file=>"logstash/agent.rb", :line=>"213", :method=>"execute"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.585000+0000", :message=>"starting pipeline", :id=>"main", :level=>:info, :file=>"logstash/agent.rb", :line=>"487", :method=>"start_pipeline"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.606000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"rubydebug", :path=>"logstash/codecs/rubydebug", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"86", :method=>"lookup"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.632000+0000", :message=>"config LogStash::Codecs::RubyDebug/@metadata = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"154", :method=>"config_init"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.789000+0000", :message=>"config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::RubyDebug metadata=>false>", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"154", :method=>"config_init"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.791000+0000", :message=>"config LogStash::Outputs::Stdout/@workers = 1", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"154", :method=>"config_init"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.796000+0000", :message=>"Will start workers for output", :worker_count=>1, :class=>"LogStash::Outputs::Stdout", :level=>:debug, :file=>"logstash/output_delegator.rb", :line=>"77", :method=>"register"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.812000+0000", :message=>"Starting pipeline", :id=>"main", :pipeline_workers=>2, :batch_size=>125, :batch_delay=>5, :max_inflight=>250, :level=>:info, :file=>"logstash/pipeline.rb", :line=>"188", :method=>"start_workers"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.816000+0000", :message=>"Pipeline main started", :file=>"logstash/agent.rb", :line=>"491", :method=>"start_pipeline"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.826000+0000", :message=>"Input plugins stopped! Will shutdown filter/output workers.", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"148", :method=>"run"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.935000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.942000+0000", :message=>"Pushing shutdown", :thread=>"#<Thread:0x6d334b22 run>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"395", :method=>"shutdown_workers"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.946000+0000", :message=>"Pushing shutdown", :thread=>"#<Thread:0x30e6a1db sleep>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"395", :method=>"shutdown_workers"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.957000+0000", :message=>"Shutdown waiting for worker thread #<Thread:0x6d334b22>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"400", :method=>"shutdown_workers"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.963000+0000", :message=>"Shutdown waiting for worker thread #<Thread:0x30e6a1db>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"400", :method=>"shutdown_workers"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.970000+0000", :message=>"closing output delegator", :klass=>"LogStash::Outputs::Stdout", :level=>:debug, :file=>"logstash/output_delegator.rb", :line=>"137", :method=>"do_close"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.972000+0000", :message=>"closing", :plugin=>"LogStash::Outputs::Stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"37", :method=>"do_close"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:31.976000+0000", :message=>"Pipeline main has been shutdown", :file=>"logstash/agent.rb", :line=>"491", :method=>"start_pipeline"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:34.823000+0000", :message=>"stopping pipeline", :id=>"main", :file=>"logstash/agent.rb", :line=>"406", :method=>"shutdown_pipelines"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:34.830000+0000", :message=>"Closing inputs", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"384", :method=>"shutdown"}
logstash_1 | {:timestamp=>"2016-10-25T13:14:34.832000+0000", :message=>"Closed inputs", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"386", :method=>"shutdown"}
這是我logstash.conf文件:
input {
}
filter {
}
output {
stdout {
codec => rubydebug # for debugging
}
}
它檢查出來,當我運行logstash -t logstash.conf
所以它不是用配置文件的問題。
問題是什麼?
cat'ing meminfo
顯示:
MemTotal: 3797716 kB
MemFree: 334368 kB
top
顯示:
top - 15:22:36 up 1 day, 1:42, 2 users, load average: 0,52, 0,53, 0,80
Tasks: 298 total, 1 running, 297 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2,1 us, 2,4 sy, 0,0 ni, 94,4 id, 0,2 wa, 0,0 hi, 0,9 si, 0,0 st
KiB Mem : 3797716 total, 325552 free, 1956392 used, 1515772 buff/cache
KiB Swap: 4063228 total, 4059964 free, 3264 used. 1610224 avail Mem
似乎並沒有成爲一個硬件問題。
一個奇怪的是從運行elasticsearch日誌消息:
elasticsearch_1 | [2016-10-25 12:37:01,076][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
但這應該不會對logstash是如何運行的任何影響,其實elasticsearch容器中運行就好了。
編輯:
docker-compose.yaml
的ELK-堆棧:
##############
## ELK STACK
##############
version: '2'
services:
elasticsearch:
image: 'elasticsearch:latest'
ports:
- "xxxx:xxxx"
networks:
- net
logstash:
build: './logstash_image'
links:
- elasticsearch
ports:
#in1
- "xxxx:xxxx"
#in2
- "xxxx:xxxx"
#in3
- "xxxx:xxxx"
#in4
- "xxxx:xxxx"
#in5
- "xxxx:xxxx"
networks:
- net
# network_mode: host # set up same network values as host (debug only)
# TODO: limit/rotate docker logs:
# https://docs.docker.com/compose/compose-file/#/logopt
# https://docs.docker.com/compose/compose-file/#logdriver
kibana:
image: 'kibana:latest'
ports:
- 'xxxx:xxxx'
links:
- elasticsearch
volumes:
- ./kibana.yml:/opt/kibana/config/kibana.yml
networks:
- net
networks:
net:
driver: bridge
ipam:
driver: default
config:
- subnet: xxx.xxx.xxx.xxx/xx
ip_range: yyy.yyy.yyy.yyy/yy
gateway: zzz.zzz.zzz.zzz
如何在'搬運工,compose.yml'配置logstash? –
@EltonStoneman我已經將它添加到問題中,必須隱藏一些實現特定的東西,但應該足以找出任何問題。撰寫文件沒有問題,並運行容器。 –