1
我第一次在Hadoop中使用自定義數據類型。這裏是我的代碼:使用Hadoop自定義數據類型時EOF異常
自定義數據類型:
public class TwitterData implements Writable {
private Long id;
private String text;
private Long createdAt;
public TwitterData(Long id, String text, Long createdAt) {
super();
this.id = id;
this.text = text;
this.createdAt = createdAt;
}
public TwitterData() {
this(new Long(0L), new String(), new Long(0L));
}
@Override
public void readFields(DataInput in) throws IOException {
System.out.println("In readFields...");
id = in.readLong();
text = in.readLine();
createdAt = in.readLong();
}
@Override
public void write(DataOutput out) throws IOException {
System.out.println("In write...");
out.writeLong(id);
out.writeChars(text);
out.writeLong(createdAt);
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getText() {
return text;
}
public void setText(String text) {
this.text = text;
}
public Long getCreatedAt() {
return createdAt;
}
public void setCreatedAt(Long createdAt) {
this.createdAt = createdAt;
}
}
映射:
public class Map extends Mapper<Object, BSONObject, Text, TwitterData>{
@Override
public void map(Object key, BSONObject value, Context context) throws IOException, InterruptedException {
BSONObject user = (BSONObject) value.get("user");
String location = (String) user.get("location");
TwitterData twitterData = new TwitterData((Long) value.get("id"),
(String) value.get("text"), (Long) value.get("createdAt"));
if(location.toLowerCase().indexOf("india") != -1) {
context.write(new Text("India"), twitterData);
} else {
context.write(new Text("Other"), twitterData);
}
}
}
主要職位代碼:
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(TwitterData.class);
我拋出映射過程後,此異常。我很害怕它爲什麼會顯示這個錯誤。誰能幫幫我嗎。 在此先感謝。
感謝托馬斯。我爲我工作。那麼當我們沒有writeLine()時,readLine()的用法是什麼?同樣,對於writeChars().. – 2013-03-07 09:34:18
@AbhendraSingh從一個文本文件中讀取並逐行讀取時。寫一行可能會寫成UTF +'\ n'。 – 2013-03-07 11:53:01