第一次運行Hadoop MapReduce字數失敗？

Question

第一次運行Hadoop字數計數示例失敗時。下面是我在做什麼：

1. 格式的NameNode：$HADOOP_HOME/bin/hdfs namenode -format

2. 開始HDFS /紗：

   $HADOOP_HOME/sbin/start-dfs.sh 
   $HADOOP_HOME/sbin/start-yarn.sh 
   $HADOOP_HOME/sbin/yarn-daemon.sh start nodemanager 
   

3. 運行單詞計數：hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar wordcount input output

（假設輸入文件夾已經在HDFS我不會把每一個通信和這裏）

輸出：

16/07/17 01:04:34 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/172.20.0.2:8032 
16/07/17 01:04:35 INFO input.FileInputFormat: Total input paths to process : 2 
16/07/17 01:04:35 INFO mapreduce.JobSubmitter: number of splits:2 
16/07/17 01:04:36 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1468688654488_0001 
16/07/17 01:04:36 INFO impl.YarnClientImpl: Submitted application application_1468688654488_0001 
16/07/17 01:04:36 INFO mapreduce.Job: The url to track the job: http://hadoop-master:8088/proxy/application_1468688654488_0001/ 
16/07/17 01:04:36 INFO mapreduce.Job: Running job: job_1468688654488_0001 
16/07/17 01:04:46 INFO mapreduce.Job: Job job_1468688654488_0001 running in uber mode : false 
16/07/17 01:04:46 INFO mapreduce.Job: map 0% reduce 0% 
Terminated 

然後HDFS崩潰，所以我不能訪問http://localhost:50070/

然後我重新啓動eveyrthing（重複步驟2），重新運行例子，一切都很好。

我該如何解決它的第一次運行？我的HDFS顯然沒有數據第一次，也許這就是問題所在？

UPDATE：

運行一個更簡單的例子也將失敗：

hadoop@8f98bf86ceba:~$ hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples*.jar pi 3 3 

Number of Maps = 3 
Samples per Map = 3 
Wrote input for Map #0 
Wrote input for Map #1 
Wrote input for Map #2 
Starting Job 
16/07/17 03:21:28 INFO client.RMProxy: Connecting to ResourceManager at hadoop-master/172.20.0.3:8032 
16/07/17 03:21:29 INFO input.FileInputFormat: Total input paths to process : 3 
16/07/17 03:21:29 INFO mapreduce.JobSubmitter: number of splits:3 
16/07/17 03:21:29 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1468696855031_0001 
16/07/17 03:21:31 INFO impl.YarnClientImpl: Submitted application application_1468696855031_0001 
16/07/17 03:21:31 INFO mapreduce.Job: The url to track the job: http://hadoop-master:8088/proxy/application_1468696855031_0001/ 
16/07/17 03:21:31 INFO mapreduce.Job: Running job: job_1468696855031_0001 
16/07/17 03:21:43 INFO mapreduce.Job: Job job_1468696855031_0001 running in uber mode : false 
16/07/17 03:21:43 INFO mapreduce.Job: map 0% reduce 0% 

同樣的問題，HDFS終止

Answer 1

您的帖子看起來不完全推斷這裏有什麼問題。我的猜測是，hadoop-mapreduce-examples-2.7.2-sources.jar是不是你想要的。更有可能你需要hadoop-mapreduce-examples-2.7.2.jar包含.class文件，而不是來源。

Answer 2

必須在MapReduce作業成功運行之前首次重新啓動HDFS。這是因爲HDFS在第一次運行時創建了一些數據，但是停止它可以清理其狀態，以便之後可以通過YARN運行MapReduce作業。

所以我的解決辦法是：

1. 開始的Hadoop：$HADOOP_HOME/sbin/start-dfs.sh
2. 停止Hadoop的：再次$HADOOP_HOME/sbin/stop-dfs.sh
3. 開始的Hadoop：$HADOOP_HOME/sbin/start-dfs.sh