Filebeat多行kubernetes容器日誌不工作

您好我有一些問題來解析kubernetes容器多行使用filebeat和logstash。 kubernetes日誌文件位於/var/log/containers/*.log和json行結構中。Filebeat多行kubernetes容器日誌不工作

有什麼關於我的配置是錯誤的？我錯過了什麼？

filebeat.yml：

filebeat: 
    # List of prospectors to fetch data. 
    prospectors: 
    - 
     paths: 
     - /var/log/containers/*.log 
     fields: {log_type: containers} 
     ignore_older: 5m 
     symlinks: true 
     json.message_key: log 
     json.keys_under_root: true 
     json.add_error_key: true 
     multiline.pattern: '^\d{4}-\d{2}-\d{2}' 
     multiline.match: after 
     multiline.negate: true 
     document_type: kube-logs 
    registry_file: /var/log/containers/filebeat_registry 
output: 
    logstash: 
    hosts: ["logstash-logging:5044"]

logstash.conf：

input { 
    beats { 
    port => 5044 
    } 
} 

filter { 
    if [type] == "kube-logs" { 

    date { 
     match => ["time", "ISO8601"] 
     remove_field => ["time"] 
    } 
    json { 
     source => "message" 
    } 

    grok { 
     match => [ "log", "<SOME_PATTERN>" ] 
     overwrite => [ "message" ] 
    } 

}

kubernetes容器：

{"log":"11:11:17,740 |-INFO in ch.qos.logback.core.joran.action.mapWAR - Attaching appender named [FILE-LOG] to Logger[ROOT]\n","stream":"stdout","time":"2017-05-09T11:11:17.742837362Z"} 
{"log":"11:11:17,740 |-INFO in ch.qos.logback.classic.joran.action.mapWAR - End of configuration.\n","stream":"stdout","time":"2017-05-09T11:11:17.742840277Z"} 
{"log":"11:11:17,741 |-INFO in ch.qos.logback.classic.joran.mapWAR - Registering current configuration as safe fallback point\n","stream":"stdout","time":"2017-05-09T11:11:17.742843277Z"} 
{"log":"\n another line","stream":"stdout","time":"2017-05-09T11:11:17.742846485Z"} 
{"log":"09-May-2017 11:11:17.756 INFO [localhost-startStop-1] org.apache.catalina.startup.mapWAR nice","stream":"stderr","time":"2017-05-09T11:11:17.756924376Z"} 
{"log":"09-May-2017 11:11:17.757 INFO [localhost-startStop-1] org.apache.catalina.startup.mapWAR great","stream":"stderr","time":"2017-05-09T11:11:17.757465828Z"}

來源

2017-05-09 Omri Ziv

我覺得你還是需要把線放在一起，你可以嘗試這個？使用{因爲日誌以{不是你的時間戳格式。

filebeat.prospectors: 
- paths: 
    - input.json 
    multiline.pattern: '^{' 
    multiline.negate: true 
    multiline.match: after 

processors: 
- decode_json_fields: 
    fields: ['message'] 
    target: json 

output.console.pretty: true

來源

2017-05-09 14:27:48

我多的問題是，我想分析 '{「日誌」：「11：11：17741 | -info在ch.qos.logback.classic.joran.mapWAR - 註冊當前配置爲安全備用\ n「，」stream「：」stdout「，」time「：」2017-05-09T11：11：17.742843277Z「} {」log「：」\ n另一行「，」stream「：」stdout「，「時間」：「2017-05-09T11：11：17.742846485Z」}' 所以我定義： 'json.message_key：日誌 multiline.pattern： '^ '」 multiline.match：後multiline.negate：true ' –

Filebeat多行kubernetes容器日誌不工作

回答

相關問題