在Ubuntu上設置單節點Hadoop集羣14.04

-4

我是Hadoop的新手，並嘗試在我的Ubuntu 14.04機器上設置單節點集羣。我遵循了Michel Nall的教程。這個羣集有一個問題......有時它會運行，有時會在沒有任何原因的情況下停止。羣集正在運行時...它不會允許我運行Java映射器和reducer。在Ubuntu上設置單節點Hadoop集羣14.04

找不到問題。任何人都可以幫助我一步一步的安裝和使用幫助。這裏的代碼和錯誤：

package org.myorg; 

import java.io.IOException; 
import java.util.*; 

import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.conf.*; 
import org.apache.hadoop.io.*; 
import org.apache.hadoop.mapred.*; 
import org.apache.hadoop.util.*; 

public class WordCount { 

    public static class Map extends MapReduceBase implements Mapper<LongWritable, Text, Text, IntWritable> { 
    private final static IntWritable one = new IntWritable(1); 
    private Text word = new Text();  
    public void map(LongWritable key, Text value, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { 
     String line = value.toString(); 
     StringTokenizer tokenizer = new StringTokenizer(line); 
     while (tokenizer.hasMoreTokens()) { 
     word.set(tokenizer.nextToken()); 
     output.collect(word, one); 
     } 
    } 
    } 

    public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> { 
    public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { 
     int sum = 0; 
     while (values.hasNext()) { 
     sum += values.next().get(); 
     } 
     output.collect(key, new IntWritable(sum)); 
    } 
    } 

    public static void main(String[] args) throws Exception { 
    JobConf conf = new JobConf(WordCount.class); 
    conf.setJobName("wordcount"); 

    conf.setOutputKeyClass(Text.class); 
    conf.setOutputValueClass(IntWritable.class); 

    conf.setMapperClass(Map.class); 
    conf.setCombinerClass(Reduce.class); 
    conf.setReducerClass(Reduce.class); 

    conf.setInputFormat(TextInputFormat.class); 
    conf.setOutputFormat(TextOutputFormat.class); 

    FileInputFormat.setInputPaths(conf, new Path(args[0])); 
    FileOutputFormat.setOutputPath(conf, new Path(args[1])); 

    JobClient.runJob(conf); 
    } 
}

錯誤是：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory 
    at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:139) 
    at WordCount.main(WordCount.java:36) 
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
    ... 2 more

來源

2015-04-01 Meenal borkar

日誌顯示什麼？你怎麼知道它沒有運行？您在所需的服務中是否有異常？ – 2015-04-01 10:19:38

Meenal，歡迎來到stackoverflow。爲了提高獲得答案的機會，請儘可能多地發佈錯誤日誌等詳細信息，以支持您所描述的每個問題。 – CKing 2015-04-01 10:21:20

這是我的代碼 – 2015-04-01 11:00:23

你得到的連接除外，它的原因是：拋出java.lang.ClassNotFoundException：org.apache.commons.logging

上級LogFactory不存在org/apache/commons/logging/path。你必須確定它是否存在。

來源

2015-04-01 12:05:55

在Ubuntu上設置單節點Hadoop集羣14.04

回答

相關問題