將來自多行日誌的信息結合到單行

我有一個java應用程序日誌文件，其中包含一些參數，我想在不同的行上選擇，我希望將它全部放在一行上。將來自多行日誌的信息結合到單行

日誌示例：

2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start 
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32 
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O) 
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end 
2017-05-31 16:30:38.891 [http-listener-1(2)] INFO com.domain.api.controller.MyDataController - Start getMyPageResponse method, input = {"token":"AppleBallCatDogElephantFoxGoatHorse"} 
2017-05-31 16:30:38.891 [http-listener-1(2)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageInfo method, requestDto = TokenRequestDto [token=AppleBallCatDogElephantFoxGoatHorse] 
2017-05-31 16:30:38.891 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - Start authentication token=AppleBallCatDogElephantFoxGoatHorse 
2017-05-31 16:30:38.895 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=123456789] 
2017-05-31 16:30:38.895 [http-listener-1(2)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 123456789, token = AppleBallCatDogElephantFoxGoatHorse 
2017-05-31 16:30:39.422 [http-listener-1(2)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageInfo method 
2017-05-31 16:30:39.423 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageResponse method 
2017-05-31 16:30:39.423 [http-listener-1(2)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageResponse method 
2017-05-31 16:30:39.423 [http-listener-1(2)] INFO com.domain.api.controller.MyDataController - End getMyPageResponse method, Total Computation/Execution Time : 0.532s 

2017-05-31 16:42:36.500 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start 
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32 
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0 
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end 
2017-05-31 16:42:36.506 [http-listener-1(3)] INFO com.domain.api.controller.MyDataController - Start getMyPageResponse method, input = {"token":"KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger"} 
2017-05-31 16:42:36.507 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - Start authentication token=KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger 
2017-05-31 16:42:36.507 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageInfo method 
2017-05-31 16:42:36.515 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=987654321] 
2017-05-31 16:42:36.515 [http-listener-1(3)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 987654321, token = KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageInfo method 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageResponse method 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageResponse method 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.api.controller.MyDataController - End getMyPageResponse method, Total Computation/Execution Time : 0.237s

過濾器實施例1：

cat /usr/local/sun/glassfish/domains/universal-api/logs/my-new-api/system.log | grep -e "Filter" -e "userID" 

2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start                          
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0              
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end                          
2017-05-31 16:30:38.895 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=123456789]                     
2017-05-31 16:30:38.895 [http-listener-1(2)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 123456789, token = AppleBallCatDogElephantFoxGoatHorse                    
2017-05-31 16:42:36.500 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start                          
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O)              
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end                          
2017-05-31 16:42:36.515 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=987654321]                     
2017-05-31 16:42:36.515 [http-listener-1(3)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 987654321, token = KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger

過濾器實施例2：

cat /usr/local/sun/glassfish/domains/universal-api/logs/my-new-api/system.log | grep -e "IP Address" -e "User-Agent" -e "userID" 

2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0              
2017-05-31 16:30:38.895 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=123456789]                     
2017-05-31 16:30:38.895 [http-listener-1(2)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 123456789, token = AppleBallCatDogElephantFoxGoatHorse                    
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O)              
2017-05-31 16:42:36.515 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=987654321]                     
2017-05-31 16:42:36.515 [http-listener-1(3)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 987654321, token = KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger

我想從這些過濾掉4兩件事：

時間，IP地址，我們ER-代理，用戶ID，令牌

樣品期望輸出：

2017-05-31 16:30:38.889, 172.25.4.32, MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0, 123456789 
2017-05-31 16:30:38.889, 172.25.4.32, Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O), 987654321

和打印他們都在同一行。

它們將始終以相同的順序出現。

，以幫助文本選擇我加入ContainerRequestContextFilter start和ContainerRequestContextFilter end

我相信grep組合，awk，sed & tr可以幫我做的工作，有什麼其他有效的方法？

更新：問題是有很多這樣的項目陸續爲多個用戶。我想知道編寫一個shell腳本，python代碼或類似的技術會更有效率。

我只能在多行日誌上使用awk獲得1行，我需要每行1行。

來源

2017-05-31 computingfreak

你做一個嘗試自己來解決這個？ – Inian

這是你想要的嗎？ 'awk -v OFS ='/ ContainerRequestContextFilter start/{time = $ 1「」$ 2; next} /客戶端IP地址/ {ipAddress = $ NF; next} /客戶端用戶代理/ {n = split（$ 0，a，「User User Agent」）; userAgent = a [n]; next}/userID =/{m = split（$ 0，b，「userID =」）; userIDAndToken = b [m]; next} END {print time ，ipAddress，userAgent，userIDAndToken}''？ –

爲什麼它應該從'12：05：11.813'開始？ – RomanPerekhrest

呆子溶液：

awk '!f && /ContainerRequestContextFilter/{ f=1; t=$1 FS $2; next } 
    /Client IP Address/{ ip=$NF; next } 
    /Client User-Agent/{ uagent=$10; for(i=11;i<=NF;i++) uagent=uagent FS $i; next } 
    /userID =.* token =/{ match($0,/userID = ([0-9]+), token = (.+)/,a); f=0; 
     printf("%s, %s, %s, %s, %s\n",t,ip,uagent,a[1],a[2]); next} 
' system.log

輸出：

2017-05-31 16:30:38.889, 172.25.4.32, Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O), 123456789, AppleBallCatDogElephantFoxGoatHorse 
2017-05-31 16:42:36.500, 172.25.4.32, MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0, 987654321, KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger

!f && /ContainerRequestContextFilter/ - 處理所述起始ContainerRequestContextFilter線
/Client IP Address/{ ip=$NF; next } - 包含加工生產線Client IP Address值
/Client User-Agent/ - 加工生產線配套Client User-Agent
/userID =.* token =/ - 加工生產線配套userID =和token =

來源

2017-05-31 10:17:45 RomanPerekhrest

我刪除了「！f && 「它的功能就像一個魅力，非常感謝！ – computingfreak

我想你已經將cat /usr/local/sun/glassfish/domains/universal-api/logs/my-new-api/system.log | grep -e "Filter" -e "userID"的輸出指向log這個文件。

然後你可以嘗試以下awk命令來實現自己的目標，

awk ' 
    /ContainerRequestContextFilter start/ {printf "%s %s,",$1,$2} 
    /Client User-Agent/ {for(i=10;i<=NF;i++) printf "%s ",$i} 
    /userID/ {printf ", %s",$9} 
    /token/ {printf " %s\n",$12} 
' log

輸出形式應爲您的要求。

來源

2017-05-31 06:41:05 CWLiu

將來自多行日誌的信息結合到單行

回答

相關問題