來自eclipse的wordcount

我在使用hadoop的eclipse插件。我可以通過製作hadoop服務器來查看HDFS中的所有文件，但是當我嘗試從eclipse運行wordcount.java文件時，它會給我例外，而從終端運行它會很順利。以下例外。來自eclipse的wordcount

2/11/14 04:09:06 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
12/11/14 04:09:06 WARN mapred.JobClient: No job jar file set. User classes may not be  found. See JobConf(Class) or JobConf#setJar(String). 
12/11/14 04:09:06 WARN snappy.LoadSnappy: Snappy native library not loaded 
12/11/14 04:09:06 INFO mapred.JobClient: Cleaning up the staging area file:/tmp/hadoop-hduser/mapred/staging/hduser1728681403/.staging/job_local_0001 
12/11/14 04:09:06 ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg 
Exception in thread "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg 
    at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:197) 
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:208) 
    at org.apache.hadoop.mapred.JobClient.writeOldSplits(JobClient.java:989) 
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:981) 
    at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174) 
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897) 
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:396) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) 
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850) 
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:824) 
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1261) 
    at WordCount.run(WordCount.java:149) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) 
    at WordCount.main(WordCount.java:155)

來源

2012-11-14 shujaat

我會用這個調查開始：

ERROR security.UserGroupInformation: PriviledgedActionException as:hduser cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/user/hduser/gutenberg

這似乎導致了問題。你確定這是正確的道路嗎？如果是這樣，您可能無權訪問它。稍後，我會嘗試儘可能消除WARN。

來源

2012-11-14 12:45:03

嗨，感謝您的回覆。問題是輸入路徑。當我給/ like/user/files/..這樣的輸入時，它不起作用，但是當我用直接URL輸入時，例如hdfs：// localhost：54310/user/...有用。我得到的錯誤是輸入路徑不存在，但它應該像hdfs：// localhost：9000/...那樣說，而我的異常就像file：// ...找不到。有任何想法嗎。再次感謝 – shujaat

謝謝Shujaat，解決了我的問題。從Eclipse我得到同樣的問題... 使用hdfs：// localhost：54310/user/...而不是「/ user/...」

來源

2013-10-18 17:30:31 Dutta

來自eclipse的wordcount

回答

相關問題