Logstash Grok過濾器讀取錯誤值

Question

我目前正在嘗試使用完整的elk堆棧（Beats-Logstash-ElasticSearch-Kibana）爲我們的應用程序設置一些數據收集。到目前爲止，一切工作，因爲它應該，但我有一個需要捕獲統計應用程序拋出的異常（例如java.lang.IllegalArgumentException）

我真的不關心堆棧跟蹤本身，所以我繼續前進僅爲該例外添加了單獨的grok過濾器。消息的

實施例：

2016-11-15 05:19:28,801 ERROR [App-Initialisation-Thread] appengine.java:520 Failed to initialize external authenticator myapp Support Access || appuser@vm23-13:/mnt/data/install/assembly app-1.4.12@cad85b224cce11eb5defa126030f21fa867b0dad 
java.lang.IllegalArgumentException: Could not check if provided root is a directory 
    at com.myapp.io.AbstractRootPrefixedFileSystem.checkAndGetRoot(AbstractRootPrefixedFileSystem.java:67) 
    at com.myapp.io.AbstractRootPrefixedFileSystem.<init>(AbstractRootPrefixedFileSystem.java:30) 
    at com.myapp.io.s3.S3FileSystem.<init>(S3FileSystem.java:32) 
    at com.myapp.io.s3.S3FileSystemDriver.loadFileSystem(S3FileSystemDriver.java:60) 
    at com.myapp.io.FileSystems.getFileSystem(FileSystems.java:55) 
    at com.myapp.authentication.ldap.S3LdapConfigProvider.initializeCloudFS(S3LdapConfigProvider.java:77) 
    at com.myapp.authentication.ldap.S3LdapConfigProvider.loadS3Config(S3LdapConfigProvider.java:51) 
    at com.myapp.authentication.ldap.S3LdapConfigProvider.getLdapConfig(S3LdapConfigProvider.java:42) 
    at com.myapp.authentication.ldap.DelegatingLdapConfigProvider.getLdapConfig(DelegatingLdapConfigProvider.java:45) 
    at com.myapp.authentication.ldap.LdapExternalAuthenticatorFactory.create(LdapExternalAuthenticatorFactory.java:28) 
    at com.myapp.authentication.ldap.LdapExternalAuthenticatorFactory.create(LdapExternalAuthenticatorFactory.java:10) 
    at com.myapp.frob.appengine.getExternalAuthenticators(appengine.java:516) 
    at com.myapp.frob.appengine.startUp(appengine.java:871) 
    at com.myapp.frob.appengine.startUp(appengine.java:754) 
    at com.myapp.jsp.KewServeInitContextListener$1.run(QServerInitContextListener.java:104) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.nio.file.NoSuchFileException: fh-ldap-config/ 
    at com.upplication.s3fs.util.S3Utils.getS3ObjectSummary(S3Utils.java:55) 
    at com.upplication.s3fs.util.S3Utils.getS3FileAttributes(S3Utils.java:64) 
    at com.upplication.s3fs.S3FileSystemProvider.readAttributes(S3FileSystemProvider.java:463) 
    at com.myapp.io.AbstractRootPrefixedFileSystem.checkAndGetRoot(AbstractRootPrefixedFileSystem.java:61) 

神交聲明的實施例：

grok { 
patterns_dir => ["./patterns"] 
match => ["message", "%{GREEDYDATA}\n%{JAVAFILE:exception}"] 
} 

測試在神交調試器顯示正確的結果：

{ 
    "GREEDYDATA": [ 
    [ 
     "2016-11-15 05:19:28,801 ERROR [App-Initialisation-Thread] appengine.java:520 Failed to initialize external authenticator myapp Support Access || appuser@vm23-13:/mnt/data/install/assembly app-1.4.12@cad85b224cce11eb5defa126030f21fa867b0dad" 
    ] 
    ], 
    "exception": [ 
    [ 
     "java.lang.IllegalArgumentException" 
    ] 
    ] 
} 

問題

當我添加的配置logstash它抓住了引起字符串，而不是例外名，即使「導致」串是層出不窮的新行字符。然而，它適用於其他異常消息，如：

016-11-15 06:17:49,691 WARN [SCReplicationWorkerThread-2] ClientJob.java:207 50345 Error communicating to server `199.181.131.249':`80'. Waiting `10' seconds before retrying... If you see this message rarely, the sc will have recovered gracefully. || appuser@vm55-12:/mnt/deployment/install/app app-3.1.23@cad85b224cce11eb5defa126030f21fa867b0dad 
java.net.SocketTimeoutException: Read timed out 
    at java.net.SocketInputStream.socketRead0(Native Method) 
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
    at java.net.SocketInputStream.read(SocketInputStream.java:170) 
    at java.net.SocketInputStream.read(SocketInputStream.java:141) 
    at java.net.SocketInputStream.read(SocketInputStream.java:223) 
    at java.io.DataInputStream.readBoolean(DataInputStream.java:242) 
    at com.myapp.replication.client.ClientJob.passCheckRevision(ClientJob.java:279) 
    at com.myapp.replication.client.ClientJob.execute(ClientJob.java:167) 
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202) 
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:534) 

任何意見，將不勝感激。

感謝

Answer 1

你有沒有設置輸入或filebeat輸入， 這樣的mutiline顯示模式與ISO8601
開始 我想，也許你mutiline未取整線

input { 
     beats { 
      port => 5044 
      codec => multiline { 
      pattern => "^[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}[\.,][0-9]{3,7} " 
      negate => true 
      what => "previous" 
      } 
      } 
     }