如何使用神交將日誌轉換成logstash

而且，日誌是這樣的：

2016-11-12 21:02:45.878|192.168.31.205|368284310235387-20|1|wangziyi|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36|GET|/api/v1|com.didi.km.api.controller.api.v1.IndexController#index[1 args]|{"hi":["asd"]}|200 
2016-11-12 21:02:46.100|192.168.31.205|368284310235387-21|1|wangziyi|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36|GET|/api/v1|com.didi.km.api.controller.api.v1.IndexController#index[1 args]|{"hi":["asd"]}|200 
2016-11-12 21:02:46.389|192.168.31.205|368284310235387-22|1|wangziyi|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36|GET|/api/v1|com.didi.km.api.controller.api.v1.IndexController#index[1 args]|{"hi":["asd"]}|200 
2016-11-12 21:02:46.507|192.168.31.205|368284310235387-23|1|wangziyi|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36|GET|/api/v1|com.didi.km.api.controller.api.v1.IndexController#index[1 args]|{"hi":["asd"]}|200

Logstash版本是5.0.0，日誌被Filebeat發在服務器上。

我logstash配置爲：

input { 
    beats { 
     port => "5043" 
    } 
} 
filter { 
    grok { 
     match => { "message" => "%{WORD:time}|%{IP:hostIP}|%{WORD:requestId}|%{NUMBER:uid:int}|%{WORD:username}|%{WORD:method}|%{URIPATHPARAM:uri}|%{WORD:cont 
rollerMethod}|%{WORD:params}" } 
    } 
} 
output { 
    stdout { codec => rubydebug } 
    elasticsearch { 
     hosts => [ "10.94.66.193:9200" ] 
    } 
}

但是，我剛纔只得到消息字段：

{ 
    "@timestamp" => 2016-11-12T13:02:48.607Z, 
     "offset" => 31831, 
     "@version" => "1", 
    "input_type" => "log", 
      "beat" => { 
     "hostname" => "localhost", 
      "name" => "localhost", 
     "version" => "5.0.0" 
    }, 
      "host" => "localhost", 
     "source" => "logs/km-access.2016-11-12.log", 
      "time" => "2016", 
     "message" => "2016-11-12 21:02:46.507|192.168.31.205|368284310235387-23|1|wangziyi|Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36|GET|/api/v1|com.didi.km.api.controller.api.v1.IndexController#index[1 args]|{\"hi\":[\"asd\"]}|200", 
      "type" => "log", 
      "tags" => [ 
     [0] "beats_input_codec_plain_applied" 
    ] 
}

有沒有在我的配置代碼中的一些問題？

來源

2016-11-12 BeeNoisy

使用追蹤圖形此日誌匹配：

%{TIMESTAMP_ISO8601}\|%{IP}\|(?<requestID>\d+-\d+)\|%{INT:uid}\|%{WORD:username}\|(?<ua>(\w|\/|\.|\s|\(|;|\)|,)+)\|%{WORD:method}\|(?<uri>(\w|\/)+)\|(?<controllerMethod>(\w|\d|\s|\.|#|\[|\])+)\|(?<param>(\w|{|}|"|\:|\[|\])+)\|%{NUMBER:statusCode}\

而且，由於https://grokdebug.herokuapp.com/

來源

2016-11-12 15:46:41 BeeNoisy

如何使用神交將日誌轉換成logstash

回答

相關問題