格式化文本文件中使用bash腳本

-1

我有一個文件（exOut.txt）按以下格式由數千行文字CSV：格式化文本文件中使用bash腳本

[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.497312, total=11.0min 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.499232, total=11.0min 
[Parallel(n_jobs=-2)]: Done 2 out of 6 | elapsed: 11.0min remaining: 22.0min 
[CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.499762, total=11.1min 
[Parallel(n_jobs=-2)]: Done 3 out of 6 | elapsed: 11.1min remaining: 11.1min 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482, score=0.449309, total=19.6min 
[Parallel(n_jobs=-2)]: Done 4 out of 6 | elapsed: 19.6min remaining: 9.8min 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482, score=0.449831, total=19.7min 
[CV] solver=newton-cg, penalty=l2, multi_class=ovr, max_iter=187.637633813, C=0.778324314482, score=0.451609, total=19.7min 
[Parallel(n_jobs=-2)]: Done 6 out of 6 | elapsed: 19.7min remaining: 0.0s 
[Parallel(n_jobs=-2)]: Done 6 out of 6 | elapsed: 19.7min finished 
...

我想寫一個shell腳本會拿這個文件並重新格式化，以csv格式創建一個新文件，只記錄具有「score」屬性的行。這應該看起來像這樣：

solver,penalty,multi_class,max_iter,C,score 
sag,l2,multinomial,187.638,0.312,0.497 
sag,l2,multinomial,187.638,0.312,0.499 
sag,l2,multinomial,187.638,0.312,0.500 
newton-cg,l2,ovr,187.638,0.779,0.449 
newton-cg,l2,ovr,187.638,0.779,0.450 
newton-cg,l2,ovr,187.638,0.779,0.450

如果可能的話，所有值四捨五入到最接近的第1000位。

最終我想借此CSV和通過識別與記錄，除了「分數」等各個領域，並與給出的參數的平均得分來取代這一個紀錄做一個濃縮版。例如：

solver,penalty,multi_class,max_iter,C,avg_score 
sag,l2,multinomial,187.638,0.312,0.499 
newton-cg,l2,ovr,187.638,0.779,0.450

任何幫助表示讚賞！我不是正則表達式的專家，主要是爲什麼我問。

編輯1個感謝您的反饋，這裏有更多的一些信息：

用grep，awk的迄今爲止我已經試過各種腳本和sed，包括grep '=.*,' exOut.txt只承認一個大發生的模式，而不是多個字段，以及僅清理每行的第一部分的sed 's/^[^\=]*\=//g' exOutput.txt > firstCSV.csv。

來源

2017-06-06 flpp

歡迎堆棧溢出。這不是一個代碼編寫服務，您可以在其中發佈您的要求和選擇的語言，並且有人爲您編寫代碼。我們非常樂意提供幫助，但我們希望您先努力自己解決問題，並將您的努力包括在您的問題中。請在這裏詢問之前[編輯]顯示你自己嘗試過的代碼。如果您需要更多信息，請參閱[問]。 –

'awk'應該很簡單。試一試，如果你仍然無法得到它，你可以發佈你的代碼和它給你的輸出的例子。（而'awk'非常值得學習。） – Jack

如果你想要一個bash腳本，爲什麼用python標記？使用scikit學習由Python程序生成 –

1.問題，你可以參考parseRawDataFile功能。

2.問題，你可以參考THR parseCsvDataFile功能。

3.In代碼有一些硬代碼，請注意。

import codecs 
datausage = r"e:\temp\111.txt" 
storagePath = r"e:\temp\222.csv" 
storagePath_New = r"e:\temp\222_New.csv" 
strStr = "score" 

def parseRawDataFile(path): 
    sumResult = "solver,penalty,multi_class,max_iter,C,score\r\n" 
    # [CV] solver=sag, penalty=l2, multi_class=multinomial, max_iter=187.637633813, C=0.31181629405, score=0.497312, total=11.0min 

    with open(path) as f: 

     lines = f.readlines() 
     for line in lines: 
      if strStr in line: 

       tempList = line.replace(",", "=").split("=") 
       # print(tempList) 
       sumResult += "%s,%s,%s,%.03f,%.03f,%.03f\r\n" % (tempList[1], tempList[3], tempList[5], float(tempList[7]), float(tempList[9]), float(tempList[11])) 

    # print(sumResult) 
    return sumResult 

def parseCsvDataFile(path): 
    sumResult = "solver,penalty,multi_class,max_iter,C,score\r\n" 
    # sag,l2,multinomial,187.638,0.312,0.497 

    nameList = [] 
    with open(path) as f: 
     lines = f.readlines() 
     temptSum = 0.0 
     tempNum = 0 
     for line in lines: 
      if "solver" in line: 
       continue 
      tempList = line.split(",") 
      if tempList[0] not in nameList: 
       if temptSum != 0: 
        sumResult += "%.03f\r\n" % (temptSum/tempNum) 
       sumResult += "%s,%s,%s,%.03f,%.03f," % (tempList[0], tempList[1], tempList[2], float(tempList[3]), float(tempList[4])) 

       tempNum = 0 
       temptSum = 0.0 
       nameList.append(tempList[0]) 
      if tempList[0] in nameList: 
       tempNum += 1 
       temptSum += float(tempList[-1]) 

     sumResult += "%.03f\r\n" % (temptSum/tempNum) 
    print(sumResult) 
    return sumResult 

def writeLog(f, d): 
    try: 
     f = codecs.open(f, 'w', 'utf-8') 
     f.write(d) 
     f.close() 
    except: 
     print("file is not exist.") 

result = parseRawDataFile(datausage) 
writeLog(storagePath, result) 
result = parseCsvDataFile(storagePath) 
writeLog(storagePath_New, result)

來源

2017-06-06 02:44:16 saul

@ cricket_007謝謝。修復了錯誤代碼格式。 – saul

格式化文本文件中使用bash腳本

回答

相關問題