阿帕奇豬腳本

Question

我要總結的RW柱的每個端口每小時

Time  ID Name    RW   
-------- --- -------   ---------- 
14:57:01 000 Port0   1340 
14:57:01 001 Port1    13 

14:58:01 000 Port0    864 
14:58:01 001 Port1    36 

14:59:01 000 Port0   1394 
14:59:01 001 Port1    22 

15:57:01 000 Port0   1340 
15:57:01 001 Port1    13 

15:58:01 000 Port0   864 
15:58:01 001 Port1    36 

15:59:01 000 Port0   1394 
15:59:01 001 Port1    22 
. 
. 
. 

20:57:01 000 Port0   1340 
20:57:01 001 Port1    13 

20:58:01 000 Port0   864 
20:58:01 001 Port1   36 

20:59:01 000 Port0   1394 
20:59:01 001 Port1    22 

我的劇本是

data = LOAD 'hdfs:/data/data.txt' USING PigStorage(',') AS (time:chararray, id:chararray, name:chararray, read:int, write:int, rw:int); 
runs = FOREACH data GENERATE time, name, rw; 

如何

Answer 1

您必須生成一個新的從時間列中調出小時，然後按小時，端口名稱分組，然後獲得每個分組的總和。

data = LOAD 'hdfs:/data/data.txt' USING PigStorage(',') AS (time:chararray, id:chararray, name:chararray, read:int, write:int, rw:int); 
runs = FOREACH data GENERATE GetHour((timestamp)time) as hour, name, rw; 
grouped = GROUP runs by (hour,name); 
port_total = FOREACH grouped GENERATE FLATTEN(group) as (hour,name),SUM(data.rw); 
DUMP port_total;