對Pig中不同的值執行計數

Question

對Pig執行看起來像兩個級別的分組時，我有一個問題。舉個例子，假設我有過一些例子輸入數據：

email_id:chararray from:chararray  to:bag{recipients:tuple(recipient:chararray)} 
e1     user1@example.com  {(friend1@example.com),(friend2@example.com),(friend3@myusers.com)} 
e2     user1@example.com  {(friend1@example.com),(friend4@example.com)} 
e3     user1@example.com  {(friend5@example.com)} 
e4     user2@example.com  {(friend2@example.com),(friend4@example.com)} 

所以每一行是從「從」用戶的電子郵件用戶（S）「到」。

我最終要的所有發件人和所有他們所發送的電子郵件的人，包括每個人發送的電子郵件＃，整理從最高到最低的列表，例如：

user1@example.com  {(friend1@example.com, 2), (friend2@example.com, 1), (friend3@example.com, 1), (friend4@example.com, 1), (friend5@example.com, 1)} 
user2@example.com  {(friend2@example.com, 1), (friend4@example.com, 1)} 

想要在豬身上解決這個問題，最好的方法是讚賞！

Answer 1

這裏是腳本的一個版本：

inpt = load '/pig_data/pig_fun/input/from_senders.txt' as (email_id:chararray, from:chararray, to:bag{recipients:tuple(recipient:chararray)}); 

pivot = foreach inpt generate from, FLATTEN(to); 
pivot = foreach pivot generate from, to::recipient as recipient; 
dump pivot; 
/* 
(user1@example.com,friend1@example.com) 
(user1@example.com,friend2@example.com) 
(user1@example.com,friend3@myusers.com) 
(user1@example.com,friend1@example.com) 
(user1@example.com,friend4@example.com) 
(user1@example.com,friend5@example.com) 
(user2@example.com,friend2@example.com) 
(user2@example.com,friend4@example.com) 
*/ 

grp = group pivot by (from, recipient); 
with_count = foreach grp generate FLATTEN(group), COUNT(pivot) as count; 
dump with_count; 
/* 
(user1@example.com,friend1@example.com,2) 
(user1@example.com,friend2@example.com,1) 
(user1@example.com,friend3@myusers.com,1) 
(user1@example.com,friend4@example.com,1) 
(user1@example.com,friend5@example.com,1) 
(user2@example.com,friend2@example.com,1) 
(user2@example.com,friend4@example.com,1) 
*/ 

to_bag = group with_count by from; 
result = foreach to_bag { 
    order_by_count = order with_count by count desc; 
    generate group as from, order_by_count.(recipient, count); 
}; 
dump result; 
/* 
(user1@example.com,{(friend1@example.com,2),(friend2@example.com,1),(friend3@myusers.com,1),(friend4@example.com,1),(friend5@example.com,1)}) 
(user2@example.com,{(friend2@example.com,1),(friend4@example.com,1)}) 
*/ 

希望它能幫助。