地圖降低輸出不正確

Question

我有一個輸入文件

UserId|TrackId|Shared|Radio|Skip 
111115|222|0|1|0 
111113|225|1|0|0 
111117|223|0|1|1 
111115|225|1|0|0 

我需要添加共享以及無線列所有軌道ID 輸出應該

222,1 
223,1 
225,2 

通過以下程序我寫的，我得到

222,1 
223,1 
225,1 
225,2. 

不知道錯誤是什麼

這是我的計劃

public class Total { 

public static class ListenMap extends Mapper<LongWritable, Text, Text, IntWritable> 
{ 
    public void map(LongWritable key, Text values, Context context) throws IOException, InterruptedException 
    { 
     String slt= values.toString(); 
     String arr[]= slt.split("[|]"); 
     String trackid= arr[1]; 
     String shared= arr[2]; 
     String radio= arr[3]; 
     int sharenum= Integer.parseInt(shared); 
     int radionum= Integer.parseInt(radio); 
     int total= sharenum+radionum; 
     context.write(new Text(trackid), new IntWritable(total)); 
    } 
} 


public static class ListenReduce extends Reducer<Text, IntWritable, Text, IntWritable> 
{ 
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException 
    { 
     int sum=0; 
     for(IntWritable x: values) 
     { 
      sum+=x.get(); 
      context.write(key, new IntWritable(sum)); 

     } 
    } 
} 
public static void main(String[] args) throws IOException, InterruptedException, ClassNotFoundException{ 
    Configuration conf= new Configuration(); 
    Job job= new Job(conf, "listen"); 

    job.setJarByClass(Total.class); 
    job.setMapperClass(ListenMap.class); 
    job.setReducerClass(ListenReduce.class); 

    job.setOutputKeyClass(Text.class); 
    job.setOutputValueClass(IntWritable.class); 

    job.setInputFormatClass(TextInputFormat.class); 
    job.setOutputFormatClass(TextOutputFormat.class); 

    FileInputFormat.addInputPath(job, new Path(args[0])); 
    FileOutputFormat.setOutputPath(job, new Path(args[1])); 

    System.exit(job.waitForCompletion(true)? 1:0); 

    } 
} 

Answer 1

在循環外部移動context.write(key, new IntWritable(sum));，除非您希望在增加它之後打印每個總和值。

我打算假設這段時間在提問時是錯字，因爲您的代碼沒有添加該代碼。

Answer 2

你寫出來的結果在for循環。將它移動到外部：

public static class ListenReduce extends Reducer<Text, IntWritable, Text, IntWritable> 
{ 
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException 
    { 
     int sum=0; 
     for(IntWritable x: values) 
     { 
      sum+=x.get(); 
     } 
     context.write(key, new IntWritable(sum)); 
    } 
} 

Answer 3

你正在爲循環寫你的上下文對象，這就是爲什麼你可以看到重複的鍵。

取而代之它應該只爲每個鍵寫入一次。

public static class ListenReduce extends Reducer<Text, IntWritable, Text, IntWritable> 
{ 
    public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException 
    { 
     int sum=0; 
     for(IntWritable x: values) 
     { 
      sum+=x.get(); 
     } 
     // Write it here 
     context.write(key, new IntWritable(sum)); 
    } 
}