JQ：使用輸入文件

Question

有條件更新/更換/添加JSON元素我收到以下輸入文件：

• input.json：

[ 
{"ID":"aaa_12301248","time_CET":"00:00:00","VALUE":10,"FLAG":"0"}, 
{"ID":"aaa_12301248","time_CET":"00:15:00","VALUE":18,"FLAG":"0"}, 
{"ID":"aaa_12301248","time_CET":"00:30:00","VALUE":160,"FLAG":"0"}, 

{"ID":"bbb_0021122","time_CET":"00:00:00","VALUE":null,"FLAG":"?"}, 
{"ID":"bbb_0021122","time_CET":"00:15:00","VALUE":null,"FLAG":"?"}, 
{"ID":"bbb_0021122","time_CET":"00:30:00","VALUE":22,"FLAG":"0"}, 

{"ID":"ccc_0021122","time_CET":"00:00:00","VALUE":null,"FLAG":"?"}, 
{"ID":"ccc_0021122","time_CET":"00:15:00","VALUE":null,"FLAG":"?"}, 
{"ID":"ccc_0021122","time_CET":"00:30:00","VALUE":20,"FLAG":"0"}, 

{"ID":"ddd_122455","time_CET":"00:00:00","VALUE":null,"FLAG":"?"}, 
{"ID":"ddd_122455","time_CET":"00:15:00","VALUE":null,"FLAG":"?"}, 
{"ID":"ddd_122455","time_CET":"00:30:00","VALUE":null,"FLAG":"?"}, 
] 

，你可以看到有一些有效值（FLAG：0）和一些無效值（FLAG：「？」）。 現在，我得到一個文件看起來像這樣（每個ID）：

aaa.json：

[ 
    {"ID":"aaa_12301248","time_CET":"00:00:00","VALUE":10,"FLAG":"0"}, 
    {"ID":"aaa_12301248","time_CET":"00:15:00","VALUE":null,"FLAG":"?"}, 
    {"ID":"aaa_12301248","time_CET":"00:55:00","VALUE":45,"FLAG":"0"} 
] 

正如你所看到的，對象一個是相同input.json但對象2無效（FLAG：「？」）。這就是爲什麼對象2必須由來自input.json的正確對象替換（值爲18）。 對象可以通過「time_CET」和「ID」元素進行標識。

此外，input.json中還會有新的對象，它們不是aaa.json的一部分等。應將這些對象添加到數組中，並且應保留aaa.json中的有效對象。

最後，aaa.json應該是這樣的：

[ 
    {"ID":"aaa_12301248","time_CET":"00:00:00","VALUE":10,"FLAG":"0"}, 
    {"ID":"aaa_12301248","time_CET":"00:15:00","VALUE":18,"FLAG":"0"}, 
    {"ID":"aaa_12301248","time_CET":"00:30:00","VALUE":160,"FLAG":"0"}, 
    {"ID":"aaa_12301248","time_CET":"00:55:00","VALUE":45,"FLAG":"0"} 
] 

因此，要總結：

1. 找FLAG： 「？」 in aaa.json
2. 使用「ID」 和「time_CET」進行映射，使用input.json中的匹配對象替換此對象。
3. 保持exisiting有效對象，並從該input.json 並未aaa.json之前存在添加對象（這意味着開始 在「ID」字段「AAA」唯一對象）
4. 重複此bbb.json ，ccc.json和ddd.json

我不確定是否有可能一次完成這個命令，因爲輸出必須返回到正確的id文件（aaa， bbb ccc.json）：

jq --argfile aaa aaa.json --argfile bbb bbb.json .... -f prog.jq input.json 

The問題是，標識符（aaa，bbb，ccc等）後面的數字可能會改變。因此，要確保對象添加到正確的文件/陣列，像這樣的語句將需要：
if (."ID"|contains("aaa")) then ....

或者是更好地運行程序多次與不同的輸入參數？我不確定..

預先感謝您！

Answer 1

這是一種方法

#!/bin/bash 

# usage: update.sh input.json aaa.json bbb.json.... 
# updates each of aaa.json bbb.json.... 

input_json="$1" 
shift 

for i in "$@"; do 
    jq -M --argfile input_json "$input_json" ' 

     # functions to restrict input.json to keys of current xxx.json file 
     def prefix:    input_filename | split(".")[0]; 
     def selectprefix:  select(.ID | startswith(prefix)); 

     # functions to build and probe a lookup table 
     def pk:     [.ID, .time_CET]; 
     def lookup($t;$k):  $t | getpath($k); 
     def lookup($t):   lookup($t;pk); 
     def organize(s):   reduce s as $r ({}; setpath($r|pk; $r)); 

     # functions to identify objects in input.json missing from xxx.json 
     def pks:     paths | select(length==2); 
     def missing($t1;$t2): [$t1|pks] - [$t2|pks] | .[]; 
     def getmissing($t1;$t2): [ missing($t1;$t2) as $p | lookup($t1;$p)]; 

     # main routine 
     organize(.[]) as $xxx 
     | organize($input_json[] | selectprefix) as $inp 
     | map(if .FLAG != "?" then . else . += lookup($inp) end) 
     | . + getmissing($inp;$xxx) 

    ' "$i" | sponge "$i" 

done 

該腳本使用JQ在循環中讀取和更新每個aaa.json ...文件。

該過濾器創建臨時對象以方便查找值[ID,time_CET]，使用FLAG ==「？」更新aaa.json中的任何值。並最終添加input.json中缺少的任何值aaa.json。

input.json的臨時查找表使用input_filename，以便只包含以當前處理文件的名稱匹配的前綴開始的密鑰。

樣品試驗：

$ ./update.sh input.json aaa.json 

aaa.json運行後：

[ 
    { 
    "ID": "aaa_12301248", 
    "time_CET": "00:00:00", 
    "VALUE": 10, 
    "FLAG": "0" 
    }, 
    { 
    "ID": "aaa_12301248", 
    "time_CET": "00:15:00", 
    "VALUE": 18, 
    "FLAG": "0" 
    }, 
    { 
    "ID": "aaa_12301248", 
    "time_CET": "00:55:00", 
    "VALUE": 45, 
    "FLAG": "0" 
    }, 
    { 
    "ID": "aaa_12301248", 
    "time_CET": "00:30:00", 
    "VALUE": 160, 
    "FLAG": "0" 
    } 
]