減少後無法運行地圖100％

-1

今天我在兩個節點的hadoop上運行了一個作業。我的程序在地圖100％後掛起，減少無法運行。返回的信息是：減少後無法運行地圖100％

14/01/15 20:30:49 INFO input.FileInputFormat: Total input paths to process : 25 
14/01/15 20:30:49 INFO util.NativeCodeLoader: Loaded the native-hadoop library 
14/01/15 20:30:49 WARN snappy.LoadSnappy: Snappy native library not loaded 
14/01/15 20:30:51 INFO mapred.JobClient: Running job: job_201401152029_0001 
14/01/15 20:30:52 INFO mapred.JobClient: map 0% reduce 0% 
14/01/15 20:30:58 INFO mapred.JobClient: map 7% reduce 0% 
14/01/15 20:31:01 INFO mapred.JobClient: map 15% reduce 0% 
14/01/15 20:31:22 INFO mapred.JobClient: map 100% reduce 0%

我的源代碼是：

PollingImage.java：

package imagePolling; 
import java.io.IOException; 
import java.util.*; 

import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.io.*; 
import org.apache.hadoop.conf.*; 

import org.apache.hadoop.mapreduce.*; 
import org.apache.hadoop.mapreduce.lib.input.*; 
import org.apache.hadoop.mapreduce.lib.output.*; 
import org.apache.hadoop.util.*; 

public class PollingImage extends Configured implements Tool{ 
public int run(String[] args) throws Exception{ 
    Job job = new Job(getConf()); 
    job.setJarByClass(PollingImage.class); 
    job.setJobName("polling_image"); 

    job.setMapperClass(PollingMapper.class); 
    job.setReducerClass(PollingReducer.class); 
    job.setMapOutputKeyClass(Text.class); 
    job.setMapOutputValueClass(Text.class); 
    job.setOutputKeyClass(Text.class); 
    job.setOutputValueClass(Text.class); 
    //job.setNumReduceTasks(0); 
    job.setInputFormatClass(TextInputFormat.class); 
    job.setOutputFormatClass(TextOutputFormat.class); 

    FileInputFormat.setInputPaths(job,new Path(args[0])); 
    FileOutputFormat.setOutputPath(job, new Path(args[1])); 

    boolean success = job.waitForCompletion(true); 

    return success?0:1; 
} 
public static void main(String[] args) { 
    try{ 
     int ret = ToolRunner.run(new PollingImage(), args); 
     System.exit(ret); 
    }catch(Exception ex){ 
     System.out.println(ex.getMessage()); 
    } 
} 
}

PollingMapper.java：

package imagePolling; 
import java.io.IOException; 

import org.apache.hadoop.fs.Path; 
import org.apache.hadoop.io.*; 
import org.apache.hadoop.mapreduce.*; 
import org.apache.hadoop.mapreduce.lib.input.*; 

public class PollingMapper extends Mapper<LongWritable, Text, Text, Text> { 
private Text nameKey=new Text(); 
public void map(LongWritable key,Text value,Context ctx) throws     IOException,InterruptedException { 
    Path filePath=((FileSplit)ctx.getInputSplit()).getPath(); 
    String name=filePath.getName().replaceAll(".txt", ""); 
    nameKey = new Text(name); 
    ctx.write(nameKey, value); 
} 
}

PollingReducer.java：

package imagePolling; 
import java.io.IOException; 
import java.util.StringTokenizer; 

import org.apache.hadoop.io.*; 
import org.apache.hadoop.mapreduce.Reducer; 

public class PollingReducer extends Reducer<Text, Text, Text, Text> { 
@Override 
public void reduce(Text key,Iterable<Text> values,Context ctx) throws IOException,InterruptedException { 
    Text v=new Text("a fool"); 
    ctx.write(key, v); 
} 
}

我已經閱讀了logs.It只是顯示地圖已經完成，沒有任何關於減少。

來源

2014-01-15 camperr

這些不是日誌幫助很大，你會在jobtracker頁面找到更多有用的日誌。通常jobtracker位於主節點上：http：// hadoop-master：50030/jobtracker.jsp – wlk

檢查您的/ etc/hosts文件並檢查IP條目againt主機名，如果其僞模式提供環回地址。

來源

2014-03-25 18:54:50 Ricky

減少後無法運行地圖100％

回答

相關問題