將豬的每個列操作的結果存儲在豬的單獨行中

Question

我需要對我的表的每一列執行一些數字操作（使用UDF）。對於每一列我都得到2個值（mean和standard-dev）。但最終的結果是(mean_1, sd_1, mean_2, sd_2, mean_3, sd_3...)，其中1,2...是列索引。但是我需要在單獨一行中的每個列的輸出。像：

mean_1, sd_1 \\for col1 
mean_2, sd_2 \\for col2 
... 

這裏是豬腳本我使用：

data = LOAD 'input_file.csv' USING PigStorage(',') AS (C0,C1,C2); 
grouped_data = GROUP data ALL; 
res = FOREACH grouped_data GENERATE FLATTEN(data), AVG(data.$1) as mean, COUNT(data.$1) as count; 
tmp = FOREACH res { 
       diff = (C1-mean)*(C1-mean); 
       GENERATE *,diff as diff; 
       }; 
grouped_diff = GROUP tmp all; 
sq_tmp = FOREACH grouped_diff GENERATE flatten(tmp), SUM(tmp.diff) as sq_sum; 
stat_tmp = FOREACH sq_tmp GENERATE mean as mean, sq_sum/count as variance, SQRT(sq_sum/count) as sd; 
stats = LIMIT stat_tmp 1; 

有誰請指導我如何實現這一目標？

Answer 1

謝謝。通過爲各個列創建均值和sd值的元組，然後將所有這些元組存儲在一個包中，我得到了所需的輸出。然後在下一步我把袋子弄平了。

tupled_stats = FOREACH raw_stats generate TOTUPLE(mean_0, var_0, sd_0) as T0, TOTUPLE(mean_1, var_1, sd_1) as T1, TOTUPLE(mean_2, var_2, sd_2) as T2; 
bagged_stats = FOREACH tupled_stats generate TOBAG(T0, T1, T2) as B; 
stats = foreach bagged_stats generate flatten(B);