打印最頻繁的行並刪除重複項

Question

我一直在試圖打印最頻繁的行，並刪除重複值的關鍵值分隔標籤在第一個字段中有很多不同的值的大文件;

例如輸入：

a|gofortheeyeboo 0.61 
a|gofortheeyeboo 0.81 
a|gofortheeyeboo 0.81 
a|gofortheeyeboo 0.81 
a|gofortheeyeboo 0.81 
a|gofortheeyeboo 0.81 
a|gofortheeyeboo 0.91 
a|gofortheeyeboo-gone 0.07 
a|gofortheeyeboo-gone 0.07 
a|gofortheeyeboo-abouttogone 0.61 
a|gofortheeyeboo-abouttogone 0.12 
b|attaack-attack  0.07 

爲不同的密鑰所需的輸出：

a|gofortheeyeboo 0.81 
a|gofortheeyeboo-gone 0.07 
a|gofortheeyeboo-abouttogone 0.61 
a|gofortheeyeboo-abouttogone 0.12 
b|attaack-attack  0.07 

到目前爲止，所管理的獲得在第二製表符分隔的字段通過刪除重複最大值輸出;

awk -F '\t' '{ if (l[$1] <= $2) l[$1] = $2} END {for (i in l) print i"\t"l[i];}' 

以上命令的輸出是不需要的;

a|gofortheeyeboo  0.91 
a|gofortheeyeboo-abouttogone 0.61 
b|attaack-attack  0.07 
a|gofortheeyeboo-gone 0.07 

Answer 1

sort input | uniq -c | sort -nr | \ 
     awk 's[$2] == $1 { print $2,$3} !s[$2] { print $2,$3; s[$2]=$1; }' 

Answer 2

keys = {} 

for line in sys.stdin: 
    line = line.strip() 

    k, v = line.split('\t') 

    if k in keys: 
     if v in keys[k]: 
      keys[k][v] += 1 
     else: 
      keys[k][v] = 1 
    else: 
     keys[k] = {v: 1} 

for k in keys: 

    items = keys[k].items() 

    # Some pair emerged more than once 
    if any(map(lambda x: x[1] > 1, items)): 
     # Calucalte max frequence 
     freq = reduce(
      lambda acc, e: acc if acc[1] > e[1] else e, 
      items 
      )[0] 
     print '{0}\t{1}'.format(k, freq) 
    # None pair emereged more than once 
    else: 
     # Print every pair 
     for v in items: 
      print '{0}\t{1}'.format(k, v[0])