logstash日期格式，沒有得到任何數據

Question

嗨以下是我在centos6 logstash服務器中的配置。我正在使用logstash 1.4.2和elasticsearch 1.2.1。我在/ var/log/messages文件和/ var /日誌/安全和有時間的格式轉發日誌是 「9月1日22點十五分34秒」

1. input.conf中

input { 
    lumberjack { 
    port => 5000 
    type => "logs" 
    ssl_certificate => "certs/logstash-forwarder.crt" 
    ssl_key => "private/logstash-forwarder.key" 
    } 
} 

2。 filter.conf

filter { 
    if [type] == "syslog" { 

grok { 
     match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } 
     add_field => [ "received_at", "%{@timestamp}" ] 
     add_field => [ "received_from", "%{host}" ] 
    } 
syslog_pri { } 
date { 
     locale => "en" // possibly this didn't work in logstash 1.4.2 
     match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss", "ISO8601"] 
     add_field => { "debug" => "timestampMatched"} 
     timezone => "UTC" 
    } 
ruby { code => "event['@timestamp'] = event['@timestamp'].getlocal"}          //I saw somewhere instead of "locale => en " we have to use this in logstash 1.4.2 
mutate { replace => [ "syslog_timestamp", "%{syslog_timestamp} +0545" ] }        //this probably won't work and give date parsing error 
} 
} 

3. output.conf

output { 
    elasticsearch { host => "logstash_server_ip" } 
    stdout { codec => rubydebug } 
} 

下面是所有客戶端服務器logstash - 轉發的conf

{ 
    "network": { 
    "servers": [ "logstash_server_ip:5000" ], 
    "timeout": 15, 
    "ssl ca": "certs/logstash-forwarder.crt" 
    }, 
    "files": [ 
    { 
     "paths": [ 
     "/var/log/messages", 
     "/var/log/secure" 
     ], 
     "fields": { "type": "syslog" } 
    } 
    ] 
} 

這是問題所在。我從5個不同時區的服務器轉發日誌，例如：EDT，NDT，NST，NPT。該logstash_server時區是在NPT（尼泊爾時間）UTC + 5:45]

所有服務器給予以下

2014/09/02 08:09:02.204882 Setting trusted CA from file: certs/logstash-forwarder.crt 
2014/09/02 08:09:02.205372 Connecting to logstash_server_ip:5000 (logstash_server_ip) 
2014/09/02 08:09:02.205600 Launching harvester on new file: /var/log/secure 
2014/09/02 08:09:02.205615 Starting harvester at position 5426763: /var/log/messages 
2014/09/02 08:09:02.205742 Current file offset: 5426763 
2014/09/02 08:09:02.279715 Starting harvester: /var/log/secure 
2014/09/02 08:09:02.279756 Current file offset: 12841221 
2014/09/02 08:09:02.638448 Connected to logstash_server_ip 
2014/09/02 08:09:09.998098 Registrar received 1024 events 
2014/09/02 08:09:15.189079 Registrar received 1024 events 

，我希望是好的，但只有一個時區NPT被轉發日誌和我能夠看到它在kibana，所有其他人給我以上的日誌，但我不能看到它在kibana。我認爲問題出現在DATE，因爲它無法解析來自不同服務器的日期。在logstash中也沒有顯示錯誤的日誌。

如何在這種情況下解決問題？

Answer 1

在logstash-轉發配置變化

"fields": { "type": "syslog" } 

到

"fields": { "type": "syslog", "syslog_timezone": "Asia/Kathmandu" } 

而變化filter.conf到

filter { 
    if [type] == "syslog" { 
    grok { 
     match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" } 
     add_field => [ "received_at", "%{@timestamp}" ] 
     add_field => [ "received_from", "%{host}" ] 
    } 
    syslog_pri { } 
    # Set timezone appropriately 
    if [syslog_timezone] in [ "Asia/Kathmandu" ] { 
     date { 
     match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] 
     remove_field => [ "syslog_timezone" ] 
     timezone => "Asia/Kathmandu" 
     } 
    } else if [syslog_timezone] in [ "America/Chicago", "US/Central" ] { 
     date { 
     match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] 
     remove_field => [ "syslog_timezone" ] 
     timezone => "America/Chicago" 
     } 
    } else if [syslog_timezone] =~ /.+/ { 
     date { 
     match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] 
     add_tag => [ "unknown_timezone" ] 
     timezone => "Etc/UTC" 
     } 
    } else { 
     date { 
     match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] 
     timezone => "Etc/UTC" 
     } 
    } 
    } 
}