從事件日誌

Question

解析出PowerShell中的CommandLine數據發送Windows事件日誌與WinLogBeat到Logstash - 主要集中在日誌中的PowerShell事件。

例子：

<'Data'>NewCommandState=Stopped SequenceNumber=1463 HostName=ConsoleHost HostVersion=5.1.14409.1005 HostId=b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a HostApplication=C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe EngineVersion=5.1.14409.1005 RunspaceId=bd4224a9-ce42-43e3-b8bb-53a302c342c9 PipelineId=167 CommandName=Import-Module CommandType=Cmdlet ScriptName= CommandPath= CommandLine=Import-Module -Verbose.\nishang.psm1<'/Data'> 

如何使用神交得到以下我提取CommandLine=場？

Import-Module -Verbose.\nishang.psm1 

Answer 1

Grok是一個正則表達式的包裝。如果你可以用正則表達式解析數據，你可以用grok來實現它。

儘管您的範圍特定於CommandLine字段，但解析大多數鍵=值日誌中的每個字段都非常簡單，並且可以爲每個字段使用一個正則表達式，並使用一些grok過濾器。如果您打算存儲，查詢和可視化日誌 - 數據越多越好。

正則表達式：

首先，我們先從以下內容：

(.*?(?=\s\w+=|\<|$)) 

• .*? - 匹配任何字符，除了行結束
• (?=\s\w+=|\<|$)) - 正預測先行斷言必須格局匹配以下
  • \s\w+= - 與在它之前的空間中，隨後是=
  • |\<|$任何單詞字符 - 可替換地可以匹配<或線路的端部，以便不包括它們的匹配組中使用。

這意味着，每場可以分析類似以下內容：

CommandLine=(.*?(?=\s\w+=|\<|$)) 

神交：

現在這意味着我們可以開始創建神交過濾器。它的強大之處在於可重用組件可能會將語義語言應用於它們。

/etc/logstash/patterns/powershell.grok：

# Patterns 
PS_KEYVALUE (.*?(?=\s\w+=|\<|$)) 

# Fields 
PS_NEWCOMMANDSTATE NewCommandState=%{PS_KEYVALUE:NewCommandState} 
PS_SEQUENCENUMBER SequenceNumber=%{PS_KEYVALUE:SequenceNumber} 
PS_HOSTNAME HostName=%{PS_KEYVALUE:HostName} 
PS_HOSTVERSION HostVersion=%{PS_KEYVALUE:HostVersion} 
PS_HOSTID HostId=%{PS_KEYVALUE:HostId} 
PS_HOSTAPPLICATION HostApplication=%{PS_KEYVALUE:HostApplication} 
PS_ENGINEVERSION EngineVersion=%{PS_KEYVALUE:EngineVersion} 
PS_RUNSPACEID RunspaceId=%{PS_KEYVALUE:RunspaceId} 
PS_PIPELINEID PipelineId=%{PS_KEYVALUE:PipelineId} 
PS_COMMANDNAME CommandName=%{PS_KEYVALUE:CommandName} 
PS_COMMANDTYPE CommandType=%{PS_KEYVALUE:CommandType} 
PS_SCRIPTNAME ScriptName=%{PS_KEYVALUE:ScriptName} 
PS_COMMANDPATH CommandPath=%{PS_KEYVALUE:CommandPath} 
PS_COMMANDLINE CommandLine=%{PS_KEYVALUE:CommandLine} 

凡%{PATTERN:label}將利用PS_KEYVALUE正則表達式，所述匹配組將與JSON該值進行標記。這是你可以靈活地命名你知道的領域。

/etc/logstash/conf.d/powershell。CONF：

input { 
    ... 
} 

filter { 
    grok { 
     patterns_dir => "/etc/logstash/patterns" 
     break_on_match => false 
     match => [ 
      "message", "%{PS_NEWCOMMANDSTATE}", 
      "message", "%{PS_SEQUENCENUMBER}", 
      "message", "%{PS_HOSTNAME}", 
      "message", "%{PS_HOSTVERSION}", 
      "message", "%{PS_HOSTID}", 
      "message", "%{PS_HOSTAPPLICATION}", 
      "message", "%{PS_ENGINEVERSION}", 
      "message", "%{PS_RUNSPACEID}", 
      "message", "%{PS_PIPELINEID}", 
      "message", "%{PS_COMMANDNAME}", 
      "message", "%{PS_COMMANDTYPE}", 
      "message", "%{PS_SCRIPTNAME}", 
      "message", "%{PS_COMMANDPATH}", 
      "message", "%{PS_COMMANDLINE}" 
     ] 
    } 
} 

output { 
    stdout { codec => "rubydebug" } 
} 

結果：

{ 
    "HostApplication" => "C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe", 
     "EngineVersion" => "5.1.14409.1005", 
     "RunspaceId" => "bd4224a9-ce42-43e3-b8bb-53a302c342c9", 
      "message" => "<'Data'>NewCommandState=Stopped SequenceNumber=1463 HostName=ConsoleHost HostVersion=5.1.14409.1005 HostId=b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a HostApplication=C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe EngineVersion=5.1.14409.1005 RunspaceId=bd4224a9-ce42-43e3-b8bb-53a302c342c9 PipelineId=167 CommandName=Import-Module CommandType=Cmdlet ScriptName= CommandPath= CommandLine=Import-Module -Verbose.\\nishang.psm1<'/Data'>", 
      "HostId" => "b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a", 
     "HostVersion" => "5.1.14409.1005", 
     "CommandLine" => "Import-Module -Verbose.\\nishang.psm1", 
     "@timestamp" => 2017-05-12T23:49:24.130Z, 
       "port" => 65134, 
     "CommandType" => "Cmdlet", 
      "@version" => "1", 
       "host" => "10.0.2.2", 
    "SequenceNumber" => "1463", 
    "NewCommandState" => "Stopped", 
     "PipelineId" => "167", 
     "CommandName" => "Import-Module", 
      "HostName" => "ConsoleHost" 
}