更容易的方法來比較變量

Question

我需要比較文件'tmpcsv2'中的一組變量與'uniq_id'中的變量，我詳細說明下面的文件。

tmpcsv2 - >該文件被另一個腳本'script1'更新，'script1'的每次運行都會更新（不附加）'tmpcsv2'中的新變量。沒有。變量可能是1，可以去錢櫃200

eg: 
2042344352 
2470697747 
2635527510 
3667769962 

uniq的-ID - >這是一個固定的變量（約100K中沒有。）

(Business Name,Job ID,Job Size) 
biz,1000036446,225210640 
biz,100006309,6710840 
biz,1000069211,2084019000 
biz,1000118720,34194040 
biz,1000150241,212322636 

我使用「的'循環+'如果'比較它們如下所示，是否有這樣做更容易或更快（影響更小）的方式？當我運行它時，需要很長時間才能輸出結果。打印命令僅用於測試，稍後將被刪除！

****Part of a bigger script**** 
amt=0 
mjc=0 
for jbid in `cat tmpcsv2` #Pick ID for match & calculation 
do 
    printf "Checking ID $jbid\n" >> Acsv3.tmp 
    for bsid in `cat uniq_id` #Matching jobs & size calulation 
    do 
     ckid=`echo $bsid | cut -d "," -f2` #ckid is the ID to check 
     jbsiz=`echo $bsid | cut -d "," -f3` #size of the ID 
     if [ $jbid == $ckid ] 
     then 
      printf "Matched at $ckid\n" #Print on Match found 
      printf "Valid -> $jbid\n" >> Bcsv3.tmp 
      ((mjc++)) #Increment Matched Job Count 
      amt=$((amt+jbsiz)) #Add size of matched jobs 
      break 
     else 
      printf "No Match at $cksid\n" #No matches 
     fi 
    done 
    printf "Check for ID $jbid done\n" >> Acsv3.tmp 
    printf "Matched $mjc jobs with combined size of $amt\n" >> Acsv3.tmp 
done 
****End of Comparision**** 

Answer 1

shell是用於處理這麼多數據的錯誤工具，但它是可行的。這裏最基本的錯誤是reading lines with for。通過在每次迭代中不重新打開文件可以顯着提高性能。

function main { 
    # Variables used elsewhere should be initialized there, not localized here. 
    typeset amt=0 mjc=0 jbid ckid jbsiz 

    while IFS= read -r jbid; do 
     printf 'Checking ID %s\n' "$jbid" >&3 
     while IFS=, read -r _ ckid jbsiz _; do 
      case $jbsiz in 
       *[^[:digit:]]*|'') 
        # validation is important for subsequent arithmetic. 
        return 1 
        ;; 
       "$ckid") # Assuming "cksid" was a typo. Replace if not. 
        printf 'Matched at %s\n' "$ckid" 
        printf 'Valid -> %s\n' "$jbid" >&4 
        ((mjc++, amt += jbsiz)) 
        break 
        ;; 
       *) 
        printf 'No match at %s\n' "$ckid" 
      esac 
     done <uniqid 
     { 
      printf 'Check for ID %s done\n' "$jbid" 
      printf 'Matched %s jobs with combined size of %s\n' "$mjc" "$amt" 
     } >&3 
    done <tmpcsv2 3>>Acsv3.tmp 4>>Bcsv3.tmp 
} 

最後，與幾乎任何其他語言一樣，最後，相當於awk腳本的性能將明顯優於此Bash腳本。通過使用mapfile而不是讀取循環，您還可以通過使用mapfile回調來模擬使用嵌套的讀取循環邏輯。

Answer 2

我想出了這個，不知道這是否可以縮短，但它確實運行得更快！任何幫助都感激不盡 ！

************ 
while read -r line #File read start 
do 
IFS=$"," 
val=$line 
amt=0 
mjc=0 
cjc=0 
for lsid in $val 
do 
    cksid=`echo $lsid | sed -e 's/*//g' -e 's/"//g'` 
    printf "Checking for $cksid\n" 
    ((cjc++)) #Count of jobs to check 
    prsnt=`grep -w $cksid uniq_id` 
    if [ $? -eq 0 ] 
    then 
     printf "Valid -> $cksid\n" 
     jbsiz=`grep -w $prsnt | cut -d, -f2` 
     ((mjc++, amt += jbsiz)) 
     break 
    else 
     printf "No Data for $cksid\n" 
    fi 

done 
done < tmpcsv2 
***********