使用awk/perl過濾tomcat日誌

-1

我想從tomcat日誌中提取錯誤|異常。由於tomcat日誌有多行錯誤細節。我想將每個新日誌條目都視爲我的記錄，即日期可能充當記錄分隔符。使用awk/perl過濾tomcat日誌

Oct 4, 2012 4:00:38 PM org.apache.catalina.loader.WebappClassLoader loadClass 
INFO: Illegal access: this web application instance has been stopped already. Could not load com.google.common.base.Stopwatch. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact. 
java.lang.IllegalStateException 
     at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531) 
     at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491) 
     at com.myweb.ontest.stats.RpcInterceptor.intercept(RpcInterceptor.java:45) 
     at com.myweb.ontest.platform.SupplySource$Iface$$EnhancerByCGLIB$$6b5e8142.finalize() 
     at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method) 
     at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83) 
     at java.lang.ref.Finalizer.access$100(Finalizer.java:14) 
     at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160) 
Oct 4, 2012 4:00:38 PM Org.apache.catalina.loader ....

請讓我知道它如何用awk/perl腳本來完成。

awk '/^[A-Z][a-z]/ && f++{print ""}1' file

，所以你可以通過使用一個空行作爲記錄分隔輕鬆地處理它在隨後的awk腳本：

感謝

來源

2012-12-02 Rajgourav Jain

我會先用添加記錄之間的空行：

awk '/^[A-Z][a-z]/ && f++{print ""}1' file | 
awk -v RS= -F'\n' '{print "Record #" NR; for (i=1;i<=NF;i++) print "Field #" i, "[" $i "]"}' 
Record #1 
Field #1 [Oct 4, 2012 4:00:38 PM org.apache.catalina.loader.WebappClassLoader loadClass] 
Field #2 [INFO: Illegal access: this web application instance has been stopped already. Could not load com.google.common.base.Stopwatch. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.] 
Field #3 [java.lang.IllegalStateException] 
Field #4 [  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)] 
Field #5 [  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)] 
Field #6 [  at com.myweb.ontest.stats.RpcInterceptor.intercept(RpcInterceptor.java:45)] 
Field #7 [  at com.myweb.ontest.platform.SupplySource$Iface$$EnhancerByCGLIB$$6b5e8142.finalize()] 
Field #8 [  at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)] 
Field #9 [  at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)] 
Field #10 [  at java.lang.ref.Finalizer.access$100(Finalizer.java:14)] 
Field #11 [  at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)] 
Record #2 
Field #1 [Oct 7, 2012 4:00:38 PM org.apache.catalina.loader.WebappClassLoader loadClass] 
Field #2 [INFO: just a dummy record] 
Field #3 [java.lang.IllegalStateException] 
Field #4 [  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1531)] 
Field #5 [  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1491)] 
Field #6 [  at com.myweb.ontest.stats.RpcInterceptor.intercept(RpcInterceptor.java:45)] 
Field #7 [  at com.myweb.ontest.platform.SupplySource$Iface$$EnhancerByCGLIB$$6b5e8142.finalize()] 
Field #8 [  at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)] 
Field #9 [  at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)] 
Field #10 [  at java.lang.ref.Finalizer.access$100(Finalizer.java:14)] 
Field #11 [  at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)]

你沒有張貼你想你的輸出看起來像這樣如果上面沒有給你足夠的信息來做到這一點，後期預期的輸出爲您的樣品輸入

來源

2012-12-02 16:26:11

而且在Perl中，你可以做這樣的： -

use strict; 
my $log = join('', <STDIN>); 
my %entries = split(/([\r\n]*\w{3} \d+, \d+ \d+:\d+:\d+ [AP]M)/m, $log); 

foreach my $key(keys %entries) 
{ 
    print "$key\n"; 
    print "$entries{$key}\n"; 
}

上面的代碼讀取從標準輸入整個日誌，將其分解的日期和保存日期，進入一個哈希並打印散列結束。

來源

2012-12-02 16:32:26 Himanshu

謝謝。 +1爲我提供了一個從日誌中提取錯誤|異常的密鑰。 –

+1？如何abt + 25 :) – Himanshu

假設你打算把這些錯誤放到MySQL數據庫中，我會這樣做。

#!/usr/bin/perl -Tw 

use strict; 
use warnings; 
use English qw(-no_match_vars $OS_ERROR); 
use Readonly; 
use DBI; 

Readonly my $LOG_FILENAME => '/var/log/tomcat.log'; 

Readonly my $DATE_REGEX => qr{ 
    \w{3} \s \d+, \s  # Oct 4, 
    \d{4}   \s  # 2012 
    \d+:\d+:\d+ \s \w{2} # 4:00:38 PM 
}xms; 

my $sth; 
{ 
    my $dbh = DBI->connect(
     'DBI:mysql:database=errors;host=localhost', 
     'error_monitor', 
     '*********', 
     { 'RaiseError' => 1 } 
    ); 

    die $DBI::errstr 
     if !$dbh; 

    $sth = $dbh->prepare_cached(q{ 
     INSERT INTO error (
      date, 
      text 
     ) VALUES (
      ?, 
      ? 
     ) 
    }); 

    die $DBI::errstr 
     if !$sth; 
} 

my $fh; 
{ 
    open $fh, '<', $LOG_FILENAME 
     or die "open $LOG_FILENAME: $OS_ERROR"; 
} 

my ($count, %record) = (0,()); 

while (my $line = <$fh>) { 

    if ($line =~ m{\A ($DATE_REGEX) \s (.+) }xms) { 

     my ($date, $text) = ($1, $2); 

     if (%record) { 
      $sth->execute(@record{qw(date text)}); 
      %record =(); 
      $count++; 
     } 

     @record{qw(date text)} = ($date, $text); 
    } 
    elsif (exists $record{date}) { 

     $record{text} .= $line; 
    } 
    else { 

     warn "malformed message: $line"; 
    } 
} 

if (%record) { 
    $sth->execute(@record{qw(date text)}); 
    $count++; 
} 

close $fh 
    or die "close $LOG_FILENAME: $OS_ERROR"; 

print "inserted $count error records\n" 
    or die "print: $OS_ERROR"; 

__END__

來源

2012-12-02 21:52:33 ddoxey

使用awk/perl過濾tomcat日誌

回答

相關問題