在hadoop中使用map減少讀取文件

-1

我想在地圖中編寫java代碼減少它在Eclipse中運行的形式，但是當我試圖在地圖中實現時減少表單我收到此錯誤請幫助我這個錯誤意味着什麼如何解決此問題在hadoop中使用map減少讀取文件

16/07/15 14:05:17 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 
16/07/15 14:05:17 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 
16/07/15 14:05:17 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized 
16/07/15 14:05:18 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this. 
16/07/15 14:05:18 INFO mapred.FileInputFormat: Total input paths to process : 2 
16/07/15 14:05:18 INFO mapreduce.JobSubmitter: Cleaning up the staging area file:/app/hadoop/tmp/mapred/staging/hadoop1148163758/.staging/job_local1148163758_0001 
Exception in thread "main" java.io.IOException: Not a file: hdfs://localhost:54310/TcTest/NewTest 
    at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:320) 
    at org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:624) 
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:616) 
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:492) 
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296) 
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) 
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293) 
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562) 
    at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) 
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557) 
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548) 
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:833) 
    at TextClassification.run(TextClassification.java:38) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
    at TextClassification.main(TextClassification.java:43) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221) 
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

來源

2016-07-15 Fazlur

_「我正在嘗試編寫java代碼」_請發佈此代碼。 – Blobonat

你需要發佈代碼，然後只有人可以提供幫助。 – a874

您需要提供輸入文件的完整路徑作爲程序的參數。右鍵單擊程序 - >運行配置 - >參數

在此之前，您需要從您的hadoop-version/share/hadoop文件夾中添加幾個jar文件。您可以參考以下博客，瞭解如何在本地模式下通過eclipse運行地圖縮減程序的完整細節。

https://acadgild.com/blog/running-mapreduce-in-local-mode-2/

來源

2016-07-15 09:57:33

我想在hadoop中運行程序並獲取這些錯誤我該如何解決它，最重要的是，爲什麼這些錯誤顯示，如果你可以解釋它將是非常有益的。 – Fazlur

這完全取決於您編寫的代碼，通過查看錯誤是很困難的。 –

在hadoop中使用map減少讀取文件

回答

相關問題