2015-11-04 40 views
1

我正試圖在EMR中部署一個Spark應用程序,並面臨以下問題。未在AWS EMR中找到SPARK_CONF

java.io.FileNotFoundException: File does not exist: hdfs://ip-10-184-176-172.ec2.internal:8020/user/hadoop/.sparkStaging/application_1446113189622_0004/__spark_conf__2712437380309904293.zip 
    at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1122) 
    at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114) 
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
    at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114) 
    at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251) 
    at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61) 
    at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359) 
    at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) 
    at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356) 
    at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
    at java.lang.Thread.run(Thread.java:745) 

我正在使用emr控制檯UI以集羣模式進行部署。在第一行中,它指定SPARK_CONF壓縮文件被上載到hdfs位置,但錯誤說在同一位置找不到文件。有沒有人遇到類似的問題?

回答

4

已解決問題。我使用的是不受支持的JAVA版本。 EMR有java 7,我的應用程序是用java 8開發的。