2016-10-25 164 views
0

正如標題所述,Logstash一旦啓動就會崩潰。
我正在通過Docker運行它。Logstash一開始崩潰

這是我的輸出:

logstash_1   | {:timestamp=>"2016-10-25T13:14:31.470000+0000", :message=>"Reading config file", :config_file=>"/etc/logstash/conf.d/logstash.conf", :level=>:debug, :file=>"logstash/config/loader.rb", :line=>"69", :method=>"local_config"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.570000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"86", :method=>"lookup"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.584000+0000", :message=>"starting agent", :level=>:info, :file=>"logstash/agent.rb", :line=>"213", :method=>"execute"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.585000+0000", :message=>"starting pipeline", :id=>"main", :level=>:info, :file=>"logstash/agent.rb", :line=>"487", :method=>"start_pipeline"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.606000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"rubydebug", :path=>"logstash/codecs/rubydebug", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"86", :method=>"lookup"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.632000+0000", :message=>"config LogStash::Codecs::RubyDebug/@metadata = false", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"154", :method=>"config_init"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.789000+0000", :message=>"config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::RubyDebug metadata=>false>", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"154", :method=>"config_init"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.791000+0000", :message=>"config LogStash::Outputs::Stdout/@workers = 1", :level=>:debug, :file=>"logstash/config/mixin.rb", :line=>"154", :method=>"config_init"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.796000+0000", :message=>"Will start workers for output", :worker_count=>1, :class=>"LogStash::Outputs::Stdout", :level=>:debug, :file=>"logstash/output_delegator.rb", :line=>"77", :method=>"register"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.812000+0000", :message=>"Starting pipeline", :id=>"main", :pipeline_workers=>2, :batch_size=>125, :batch_delay=>5, :max_inflight=>250, :level=>:info, :file=>"logstash/pipeline.rb", :line=>"188", :method=>"start_workers"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.816000+0000", :message=>"Pipeline main started", :file=>"logstash/agent.rb", :line=>"491", :method=>"start_pipeline"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.826000+0000", :message=>"Input plugins stopped! Will shutdown filter/output workers.", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"148", :method=>"run"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.935000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.942000+0000", :message=>"Pushing shutdown", :thread=>"#<Thread:0x6d334b22 run>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"395", :method=>"shutdown_workers"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.946000+0000", :message=>"Pushing shutdown", :thread=>"#<Thread:0x30e6a1db sleep>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"395", :method=>"shutdown_workers"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.957000+0000", :message=>"Shutdown waiting for worker thread #<Thread:0x6d334b22>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"400", :method=>"shutdown_workers"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.963000+0000", :message=>"Shutdown waiting for worker thread #<Thread:0x30e6a1db>", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"400", :method=>"shutdown_workers"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.970000+0000", :message=>"closing output delegator", :klass=>"LogStash::Outputs::Stdout", :level=>:debug, :file=>"logstash/output_delegator.rb", :line=>"137", :method=>"do_close"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.972000+0000", :message=>"closing", :plugin=>"LogStash::Outputs::Stdout", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"37", :method=>"do_close"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:31.976000+0000", :message=>"Pipeline main has been shutdown", :file=>"logstash/agent.rb", :line=>"491", :method=>"start_pipeline"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:34.823000+0000", :message=>"stopping pipeline", :id=>"main", :file=>"logstash/agent.rb", :line=>"406", :method=>"shutdown_pipelines"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:34.830000+0000", :message=>"Closing inputs", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"384", :method=>"shutdown"} 
logstash_1   | {:timestamp=>"2016-10-25T13:14:34.832000+0000", :message=>"Closed inputs", :level=>:info, :file=>"logstash/pipeline.rb", :line=>"386", :method=>"shutdown"} 

這是我logstash.conf文件:

input { 

} 

filter { 

} 

output { 
    stdout { 
     codec => rubydebug # for debugging 
    } 
} 

它檢查出來,當我運行logstash -t logstash.conf所以它不是用配置文件的問題。

問題是什麼?
cat'ing meminfo顯示:

MemTotal:  3797716 kB 
MemFree:   334368 kB 

top顯示:

top - 15:22:36 up 1 day, 1:42, 2 users, load average: 0,52, 0,53, 0,80 
Tasks: 298 total, 1 running, 297 sleeping, 0 stopped, 0 zombie 
%Cpu(s): 2,1 us, 2,4 sy, 0,0 ni, 94,4 id, 0,2 wa, 0,0 hi, 0,9 si, 0,0 st 
KiB Mem : 3797716 total, 325552 free, 1956392 used, 1515772 buff/cache 
KiB Swap: 4063228 total, 4059964 free,  3264 used. 1610224 avail Mem 

似乎並沒有成爲一個硬件問題。
一個奇怪的是從運行elasticsearch日誌消息:
elasticsearch_1 | [2016-10-25 12:37:01,076][WARN ][bootstrap ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
但這應該不會對logstash是如何運行的任何影響,其實elasticsearch容器中運行就好了。

編輯:
docker-compose.yaml的ELK-堆棧:

############## 
## ELK STACK 
############## 

version: '2' 

services: 
    elasticsearch: 
    image: 'elasticsearch:latest' 
    ports: 
     - "xxxx:xxxx" 
    networks: 
     - net 

    logstash: 
    build: './logstash_image' 
    links: 
     - elasticsearch 
    ports: 
    #in1 
     - "xxxx:xxxx" 
    #in2 
     - "xxxx:xxxx" 
    #in3 
     - "xxxx:xxxx" 
    #in4 
     - "xxxx:xxxx" 
    #in5 
     - "xxxx:xxxx" 
    networks: 
     - net 
    # network_mode: host # set up same network values as host (debug only) 
# TODO: limit/rotate docker logs: 
#  https://docs.docker.com/compose/compose-file/#/logopt 
#  https://docs.docker.com/compose/compose-file/#logdriver 

    kibana: 
    image: 'kibana:latest' 
    ports: 
     - 'xxxx:xxxx' 
    links: 
     - elasticsearch 
    volumes: 
     - ./kibana.yml:/opt/kibana/config/kibana.yml 
    networks: 
     - net 


networks: 
    net: 
    driver: bridge 
    ipam: 
     driver: default 
     config: 
     - subnet: xxx.xxx.xxx.xxx/xx 
     ip_range: yyy.yyy.yyy.yyy/yy 
     gateway: zzz.zzz.zzz.zzz 
+0

如何在'搬運工,compose.yml'配置logstash? –

+0

@EltonStoneman我已經將它添加到問題中,必須隱藏一些實現特定的東西,但應該足以找出任何問題。撰寫文件沒有問題,並運行容器。 –

回答

4

Logstash不崩潰,它只是退出,因爲您的配置文件中明確指定了一個空輸入源(input {})。日誌管道沒有任何類型的數據源(默認情況下它至少將STDIN作爲數據源),所以它只是退出。

注意input {}明確限定一個空的輸入是不同而不是限定任何輸入(在這種情況下,Logstash將假定input { stdin { type => stdin } }作爲缺省值和偵聽STDIN流數據輸入)。使用給定的字符串作爲配置數據 -

-e CONFIG_STRING

此行爲Logstash的幫助輸出(logstash agent --help)(重點煤礦)定義。與配置文件相同的語法。 如果沒有指定輸入,再下面是用作默認輸入input { stdin { type => stdin } } [...]

+0

我只使用了我提供的準系統'logstash.conf'作爲例子,實際上我使用了一個包含5個輸入源,2個過濾器和1個輸出的配置。看起來我在docker-compose rebuild中做了一些錯誤,所以我的新配置從未被複制過,因此錯誤的配置已經到位。感謝您澄清這一點並幫助我排除故障。 –

+0

順便說一句,如何使用'logstash agent'命令與僅使用'logstash'命令不同。 'logstash -e foo'? Afaik'logstash agent'是logstash使用的運行時,但我找不到太多的文檔。 –