錯誤：java.io.IOException：錯誤的值類：類org.apache.hadoop.io.Text不是類Myclass

我有我的映射器和reducer如下。但我正在得到某種奇怪的例外。我不明白爲什麼它會拋出這樣的例外。錯誤：java.io.IOException：錯誤的值類：類org.apache.hadoop.io.Text不是類Myclass

public static class MyMapper implements Mapper<LongWritable, Text, Text, Info> { 

    @Override 
    public void map(LongWritable key, Text value, 
     OutputCollector<Text, Info> output, Reporter reporter) 
     throws IOException { 
     Text text = new Text("someText") 
      //process 
     output.collect(text, infoObjeject); 
    } 

} 

public static class MyReducer implements Reducer<Text, Info, Text, Text> { 

    @Override 
    public void reduce(Text key, Iterator<Info> values, 
     OutputCollector<Text, Text> output, Reporter reporter) 
     throws IOException { 
     String value = "xyz" //derived in some way 
     //process 
     output.collect(key, new Text(value)); //exception occurs at this line 
    } 

} 

System.out.println("Starting v14 "); 
JobConf conf = new JobConf(RouteBuilderJob.class); 
conf.setJobName("xyz"); 

String jarLocation =ClassUtil.findContainingJar(getClass()); 

System.out.println("path of jar file = " + jarLocation); 

conf.setJarByClass(RouteBuilderJob.class); 

conf.setMapOutputKeyClass(Text.class); 
conf.setMapOutputValueClass(Info.class); 

conf.setOutputKeyClass(Text.class); 
conf.setOutputValueClass(Text.class); 

//am i missing something here??? 

conf.setMapperClass(RouteBuilderJob.RouteMapper.class); 
conf.setCombinerClass(RouteBuilderJob.RouteReducer.class); 
conf.setReducerClass(RouteBuilderJob.RouteReducer.class); 


conf.setInputFormat(TextInputFormat.class); 
conf.setOutputFormat(TextOutputFormat.class); 

FileInputFormat.setInputPaths(conf, new Path(args[0])); 
FileOutputFormat.setOutputPath(conf, new Path(args[1])); 

JobClient.runJob(conf);

我得到一個異常：

Error: java.io.IOException: wrong value class: class org.apache.hadoop.io.Text is not class com.xyz.mypackage.Info 
at org.apache.hadoop.mapred.IFile$Writer.append(IFile.java:199) 
at org.apache.hadoop.mapred.Task$CombineOutputCollector.collect(Task.java:1307) 
at com.xyz.mypackage.job.MyJob$RouteReducer.reduce(MyJob.java:156) 
at com.xyz.mypackage.job.MyJob$RouteReducer.reduce(MyJob.java:1)

內部信息對象（實現Writable）進行序列化使用Text

@Override 
public void write(DataOutput out) throws IOException { 
    Gson gson = new Gson(); 
    String searlizedStr = gson.toJson(this); 
    Text.writeString(out, searlizedStr); 
} 

@Override 
public void readFields(DataInput in) throws IOException { 
    String s = Text.readString(in); 
    Gson gson = new Gson(); 
    JsonReader jsonReader = new JsonReader(new StringReader(s)); 
    jsonReader.setLenient(true); 

Info info = gson.fromJson(jsonReader, Info.class); 
    //set fields using this.somefield = info.getsomefield() 
}

來源

2014-01-27 user93796

什麼是「價值」在這裏。我沒有找到任何聲明或任何「價值」變量？「output.collect（key，new Text（value））;」在這一行 – Backtrack

@Backtrack的值是一個字符串 – user93796

@Backtrack你能幫我這個嗎？我花了很多時間調試這個 – user93796

技術上輸出類型有你的減少應該是相同的作爲你的輸入類型。如果您使用組合器作爲組合器的輸出送入您的減速器，則必須如此。

來源

2014-05-04 16:55:17 b4hand

救了我一天@ b4hand –

我也是:)謝謝！ – Leo

也幫助了我。但是減速器爲什麼會希望這兩種類型保持不變的具體原因 – Abhi

錯誤：java.io.IOException：錯誤的值類：類org.apache.hadoop.io.Text不是類Myclass

回答

相關問題