2010-07-24 52 views
7

我使用Hadoop在Windows和我已經配置好的一切(安裝了cygwin,密碼的ssh等)Hadoop的Windows中:文件中未發現異常

我編譯在WC.jar的單詞計數程序並試圖跑步。它在獨立模式下完美運行..但在完全分佈模式下,它給出FileNotFoundException

請查看日誌並告訴我它有什麼問題。 我已經開始了MACH1中的dfs和mapreduce。 (那是我的主人)

$ bin/hadoop jar WC.jar WordCount words result 

10/07/24 16:57:38 INFO input.FileInputFormat: Total input paths to process : 2 
10/07/24 16:57:39 INFO mapred.JobClient: Running job: job_201007241657_0001 
10/07/24 16:57:40 INFO mapred.JobClient: map 0% reduce 0% 
10/07/24 16:57:50 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00003_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:57:55 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_r_0 
00002_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_r_000002_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:07 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00003_1, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_1/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:14 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00003_2, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_2/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:26 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00002_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:34 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_r_0 
00001_0, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_r_000001_0/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:41 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00002_1, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_1/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:47 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 
00002_2, Status : FAILED 
java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke 
r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_2/work/tmp d 
oes not exist. 
     at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys 
tem.java:361) 
     at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. 
java:245) 
     at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) 

     at org.apache.hadoop.mapred.Child.main(Child.java:155) 

10/07/24 16:58:53 INFO mapred.JobClient: Job complete: job_201007241657_0001 
10/07/24 16:58:53 INFO mapred.JobClient: Counters: 0 

[email protected] /usr/local/hadoop-0.20.2 
$` 

謝謝。

+0

向我們展示[SSCCE](http://sscce.org/)代碼,您可以在其中調用'DistributedCache.addCacheFile'。請參閱[這裏](http://stackoverflow.com/questions/3219012/filenotfoundexception-when-using-hadoop-distributed-cache)以獲得可能的答案。 – opyate 2011-06-30 22:42:35

回答

1

我想我可能以前見過這個例外,但我沒有訪問我的舊日誌來確認它。我通過解決了我的FileNotFoundException異常,重新命名了名稱節點。您可能需要檢查名稱節點日誌中的「不一致狀態」以在重新格式化之前確認原因。

+0

hi @wei,我也有同樣的問題,我也嘗試過重新格式化,但問題仍然存在 – jerith 2013-04-11 06:21:10

+0

你在日誌中看到「不一致的狀態」嗎? – Wei 2013-04-14 11:37:09