使用JQ

Question

以JSON不同值的唯一組合我有一個JSON文件（input.json），它看起來是這樣的：

{"header1":"a","header2":1a, "header3":1a, "header4":"apple"}, 
{"header1":"b","header2":2a, "header3":2a, "header4":"orange"} 
{"header1":"c","header2":1a, "header3":2a, "header4":"banana"}, 
{"header1":"d","header2":2a, "header3":1a, "header4":"apple"}, 
{"header1":"a","header2":2a, "header3":1a, "header4":"banana"}, 
{"header1":"b","header2":1a, "header3":2a, "header4":"orange"}, 
{"header1":"b","header2":1a, "header3":1a, "header4":"orange"}, 
{"header1":"d","header2":1a, "header3":1a, "header4":"apple"}, 
{"header1":"a","header2":2a, "header3":1a, "header4":"banana"} (repeat of line 5) 

我想過濾出的每個值JQ的獨特組合。 結果應該是這樣的：

{"header1":"a","header2":1a, "header3":1a, "header4":"apple"}, 
{"header1":"b","header2":2a, "header3":2a, "header4":"orange"} 
{"header1":"c","header2":1a, "header3":2a, "header4":"banana"}, 
{"header1":"d","header2":2a, "header3":1a, "header4":"apple"}, 
{"header1":"a","header2":2a, "header3":1a, "header4":"banana"}, 
{"header1":"b","header2":1a, "header3":2a, "header4":"orange"}, 
{"header1":"b","header2":1a, "header3":1a, "header4":"orange"}, 
{"header1":"d","header2":1a, "header3":1a, "header4":"apple"} 

我試圖通過與其他頭頭1的做組，但它並沒有產生獨特的效果。 我用unique但沒有產生正確的結果。

我怎樣才能得到這個？ Im新的jq和沒有找到很多教程。

感謝

Answer 1

1. 你給採樣線是無效的JSON。由於您的序言將它們作爲JSON引入，因此以下內容將假定您打算呈現JSON對象的數組。

2. 的問題是，在許多方面還不清楚，但是從例子中，它看起來好像unique可能是你在找什麼，所以考慮：

調用：JQ -c「獨特的[ ]」 input.json

輸出：

{"header1":"a","header2":"1a","header3":"1a","header4":"apple"} 
{"header1":"a","header2":"2a","header3":"1a","header4":"banana"} 
{"header1":"b","header2":"1a","header3":"1a","header4":"orange"} 
{"header1":"b","header2":"1a","header3":"2a","header4":"orange"} 
{"header1":"b","header2":"2a","header3":"2a","header4":"orange"} 
{"header1":"c","header2":"1a","header3":"2a","header4":"banana"} 
{"header1":"d","header2":"1a","header3":"1a","header4":"apple"} 
{"header1":"d","header2":"2a","header3":"1a","header4":"apple"} 

如果需要以一些其它形式的輸出，可以d Ø在使用JQ爲好，但要求不是那麼清楚，讓我們離開，作爲一個練習:-)

Answer 2

由於作爲peak表明了自己的輸入是不合法的JSON我已經採取了糾正的自由它並轉換到個體對象的列表：

{"header1":"a","header2":"1a", "header3":"1a", "header4":"apple"} 
{"header1":"b","header2":"2a", "header3":"2a", "header4":"orange"} 
{"header1":"c","header2":"1a", "header3":"2a", "header4":"banana"} 
{"header1":"d","header2":"2a", "header3":"1a", "header4":"apple"} 
{"header1":"a","header2":"2a", "header3":"1a", "header4":"banana"} 
{"header1":"b","header2":"1a", "header3":"2a", "header4":"orange"} 
{"header1":"b","header2":"1a", "header3":"1a", "header4":"orange"} 
{"header1":"d","header2":"1a", "header3":"1a", "header4":"apple"} 
{"header1":"a","header2":"2a", "header3":"1a", "header4":"banana"} 

如果此數據是在data.json和運行

jq -M -s -f filter.jq data.json 

具有以下filter.jq

foreach .[] as $r (
    {} 
; ($r | map(.)) as $p | if getpath($p) then empty else setpath($p;1) end 
; $r 
) 

它將以原始順序生成以下輸出，但不會有重複項。

{"header1":"a","header2":"1a","header3":"1a","header4":"apple"} 
{"header1":"b","header2":"2a","header3":"2a","header4":"orange"} 
{"header1":"c","header2":"1a","header3":"2a","header4":"banana"} 
{"header1":"d","header2":"2a","header3":"1a","header4":"apple"} 
{"header1":"a","header2":"2a","header3":"1a","header4":"banana"} 
{"header1":"b","header2":"1a","header3":"2a","header4":"orange"} 
{"header1":"b","header2":"1a","header3":"1a","header4":"orange"} 
{"header1":"d","header2":"1a","header3":"1a","header4":"apple"} 

注意，

($r | map(.)) 

用於產生從被假設爲總是產生一個唯一的密鑰的路徑的每一行 僅包含值的數組。對於樣本數據 是這樣，但對於更復雜的值可能不是這樣。

較慢但是更強大的filter.jq是

foreach .[] as $r (
    {} 
; [$r | tojson] as $p | if getpath($p) then empty else setpath($p;1) end 
; $r 
) 

其使用整行的JSON表示作爲一個獨特的密鑰，以確定是否一個行先前已經看到。