問題 - 優文庫

我已經創建以下映射器和減速器使用Mahout的問題

package mypackage.ItemSimilarity; 

import java.io.IOException; 
import java.util.regex.Matcher; 
import java.util.regex.Pattern; 

import org.apache.hadoop.io.LongWritable; 
import org.apache.hadoop.io.Text; 
import org.apache.hadoop.mapreduce.Mapper; 
import org.apache.mahout.math.VarLongWritable; 

public class ItemPrefMapper extends 
     Mapper<LongWritable, Text, VarLongWritable, VarLongWritable> { 

    private static final Pattern NUMBERS = Pattern.compile("(\\d+)"); 

    @Override 
    public void map(LongWritable key, Text value, Context context) 
      throws IOException, InterruptedException { 
     String line = value.toString(); 
     Matcher m = NUMBERS.matcher(line); 
     m.find(); 
     VarLongWritable userID = new VarLongWritable(Long.parseLong(m.group())); 
     VarLongWritable itemID = new VarLongWritable(); 
     while (m.find()) { 
      itemID.set(Long.parseLong(m.group())); 
      context.write(userID, itemID); 
     } 
    } 
}

減少類

package mypackage.ItemSimilarity; 

import java.io.IOException; 

import org.apache.hadoop.mapreduce.Reducer; 
import org.apache.mahout.math.RandomAccessSparseVector; 
import org.apache.mahout.math.VarLongWritable; 
import org.apache.mahout.math.Vector; 
import org.apache.mahout.math.VectorWritable; 

public class UserVectorReducer 
     extends 
     Reducer<VarLongWritable, VarLongWritable, VarLongWritable, VectorWritable> { 
    @Override 
    public void reduce(VarLongWritable userID, 
      Iterable<VarLongWritable> itemPrefs, Context context) 
      throws IOException, InterruptedException { 
     Vector userVector = new RandomAccessSparseVector(Integer.MAX_VALUE, 100); 
     for (VarLongWritable itemPref : itemPrefs) { 
      userVector.set((int) itemPref.get(), 1.0f); 
     } 
     context.write(userID, new VectorWritable(userVector)); 
    } 

}

Spring配置文件來運行這個

<job id="mahoutJob" input-path="/home/ubuntu/input/data.txt" output-path="/home/ubuntu/output" 
mapper="mypackage.ItemSimilarity.ItemPrefMapper" 
reducer="mypackage.ItemSimilarity.UserVectorReducer" 
jar-by-class="mypackage.ItemSimilarity.ItemPrefMapper"/> 

<job-runner id="myjob-runner" pre-action="setupScript" job-ref="mahoutJob" 
run-at-startup="true"/>

當我運行這個我得到了以下錯誤。我擴展了Hadoop映射器類，但是Spring說它不是映射器類。

了java.lang.RuntimeException：類mypackage.ItemSimilarity.ItemPrefMapper不org.apache.hadoop.mapreduce.Mapper 在org.apache.hadoop.conf.Configuration.setClass（Configuration.java:931）在org.springframework.data.hadoop.mapreduce.JobFactoryBean.afterPropertiesSet（JobFactoryBean.java:153） org.springframework.beans.factory中的org.apache.hadoop.mapreduce.Job.setMapperClass（Job.java:175）。 support.AbstractAutowireCapableBeanFactory.invokeInitMethods（AbstractAutowireCapableBeanFactory.java:1571） at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean（AbstractAutowireCapableBeanFactory.java:1509）在org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean（AbstractAutowireCapableBeanFactory.java:521）在org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean（AbstractAutowireCapableBeanFactory.java:458）

來源

2014-02-13 Tarun Nagpal

你關於你的jar-by-class元素？因爲它應該指向你實例化你的ApplicationContext實例的主要方法。

另外，你確定你的軟件包名稱嗎？

com.threepillar.labs.ItemSimilarity.ItemPrefMapper

和

mypackage.ItemSimilarity.ItemPrefMapper

來源

2014-02-13 14:48:33 Chiron

這不是一個問題，因爲給定的映射類是正裝但它說這個映射器不是一個hadoop映射器，儘管我正在從hadoop映射器擴展我的映射器。 –

@TarunNagpal檢查你的軟件包名稱 – Chiron

它只是在堆棧溢出構成錯字，所以我正在編輯我的帖子 –

問題

回答

相關問題