輸入數據文件到logstash

Question

我有一個csv文件，用下面的標題：

"PacketId","MACAddress","Date","PacketLength","SourceIP","SourcePort","DestIP","DestPort" 

我想指數使用LogStash的數據ElasticSearch，而不能寫同一個過濾器。

filter { 
    grok { 
     match => message => "%{IP:SourceIP}" 
    } 
    } 

上述過濾器給SOURCEIP領域的一個很好的提取，但我怎麼寫神交模式來提取它的所有領域。

Answer 1

讓下面的CSV文件：

1,00-14-22-01-23-45,13/09/2015,32,128.248.1.43,9980,128.248.23.13,9880 
1,01-74-02-84-13-98,14/09/2015,64,128.248.1.94,9280,128.248.13.84,9380 

這裏Logstash配置必須設置：

input { 
    file { 
     path => "/path/of/your/csv/test.csv" 
     sincedb_path => "/path/of/your/csv/test.idx" 
     start_position => "beginning" 
    } 
} 

filter { 
    csv { 
     separator => "," 
     columns => ["PacketId","MACAddress","Date","PacketLength","SourceIP","SourcePort","DestIP","DestPort"] 
    } 
} 

output { 
    stdout { 
     codec => rubydebug  
    } 
} 

您將在輸出結果得到：

{ 
     "message" => [ 
     [0] "1,00-14-22-01-23-45,13/09/2015,32,128.248.1.43,9980,128.248.23.13,9880" 
    ], 
     "@version" => "1", 
     "@timestamp" => "2015-09-14T20:11:28.976Z", 
      "host" => "MyHost.local", 
      "path" => "/path/of/your/csv/test.csv", 
     "PacketId" => "1", 
     "MACAddress" => "00-14-22-01-23-45", 
      "Date" => "13/09/2015", 
    "PacketLength" => "32", 
     "SourceIP" => "128.248.1.43", 
     "SourcePort" => "9980", 
      "DestIP" => "128.248.23.13", 
     "DestPort" => "9880" 
} 
{ 
     "message" => [ 
     [0] "1,01-74-02-84-13-98,14/09/2015,64,128.248.1.94,9280,128.248.13.84,9380" 
    ], 
     "@version" => "1", 
     "@timestamp" => "2015-09-14T20:11:28.978Z", 
      "host" => "MyHost.local", 
      "path" => "/path/of/your/csv/test.csv", 
     "PacketId" => "1", 
     "MACAddress" => "01-74-02-84-13-98", 
      "Date" => "14/09/2015", 
    "PacketLength" => "64", 
     "SourceIP" => "128.248.1.94", 
     "SourcePort" => "9280", 
      "DestIP" => "128.248.13.84", 
     "DestPort" => "9380" 
} 

問候， 阿蘭

Answer 2

您需要先使用CSV filter，而不是grok。