2013-11-14 62 views
0

我一直在測試單節點上的映射reduce作業,它似乎可行,但現在我試圖在遠程集羣上運行它,我正在獲取ClassNotFoundExcepton。在羣集上運行HBase映射reduce作業時發生ClassNotFoundException

public class Pivot { 
    public static class Mapper extends TableMapper<ImmutableBytesWritable, ImmutableBytesWritable> { 
     @Override 
     public void map(ImmutableBytesWritable rowkey, Result values, Context context) throws IOException { 
      (map code) 
     } 
    } 

    public static class Reducer extends TableReducer<ImmutableBytesWritable, ImmutableBytesWritable, ImmutableBytesWritable> { 
     public void reduce(ImmutableBytesWritable key, Iterable<ImmutableBytesWritable> values, Context context) throws IOException, InterruptedException { 
      (reduce code) 
     } 
    } 

    public static void main(String[] args) { 
     Configuration conf = HBaseConfiguration.create(); 
     conf.set("fs.default.name", "hdfs://hadoop-master:9000"); 
     conf.set("mapred.job.tracker", "hdfs://hadoop-master:9001"); 
     conf.set("hbase.master", "hadoop-master:60000"); 
     conf.set("hbase.zookeeper.quorum", "hadoop-master"); 
     conf.set("hbase.zookeeper.property.clientPort", "2222"); 
     Job job = new Job(conf); 
     job.setJobName("Pivot"); 
     job.setJarByClass(Pivot.class); 
     Scan scan = new Scan(); 
     TableMapReduceUtil.initTableMapperJob("InputTable", scan, Mapper.class, ImmutableBytesWritable.class, ImmutableBytesWritable.class, job); 
     TableMapReduceUtil.initTableReducerJob("OutputTable", Reducer.class, job); 
     job.waitForCompletion(true); 
    } 
} 

當我嘗試運行此作業我收到的錯誤是::

java.lang.RuntimeException: java.lang.ClassNotFoundException: Pivot$Mapper 
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857) 
    ... 

有我丟失的東西我的代碼結構如下?爲什麼這項工作很難找到映射器?

+0

你是如何調用工作運行(請提供使用完整的命令行) –

+0

我跑它從日食 – user2623855

回答

0

從Eclipse運行作業時,需要注意Hadoop要求您從jar中啓動作業。 Hadoop需要這樣才能將代碼發送到HDFS/JobTracker。

在你的情況下,我想你沒有將你的工作類捆綁到jar中,然後從jar中運行程序 - 導致一個CNFE。

嘗試建立一個罐子,使用hadoop jar myjar.jar ...在命令行中運行,一旦這個工作,那麼你可以測試在Eclipse

相關問題