Hadoop紗線工作：錯誤的FS

我安裝了一個cloudera羣集與流浪箱。我得到一個錯誤，當我啓動下面的例子：Hadoop紗線工作：錯誤的FS

hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep input output23 'dfs[a-z.]+'

我去檢查日誌中/var/log/hadoop-yarn。有幾個日誌文件，在yarn-yarn-nodemanager-cdh-master.log，有以下stackstrace：

2015-06-17 11:42:42,398 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1434535025160_0001_000001 (auth:SIMPLE) 
2015-06-17 11:42:42,597 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1434535025160_0001_01_ 
000001 by user vagrant 
2015-06-17 11:42:42,762 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app appli 
cation_1434535025160_0001 
2015-06-17 11:42:42,776 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1434535025160_0001 tran 
sitioned from NEW to INITING 
2015-06-17 11:42:42,778 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=vagrant  IP=10.10.50.5 OPERATION=Start Container Request 
TARGET=ContainerManageImpl  RESULT=SUCCESS APPID=application_1434535025160_0001 CONTAINERID=container_1434535025160_0001_01_000001 
2015-06-17 11:42:43,997 FATAL org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread 
java.lang.IllegalArgumentException: Wrong FS: hdfs://var/log/hadoop-yarn, expected: hdfs://cdh-master:8020 
     at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1128) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1124) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1124) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:192) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:319) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:443) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:67) 
     at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173) 
     at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106) 
     at java.lang.Thread.run(Thread.java:744) 
2015-06-17 11:42:44,000 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Adding container_1434535025160_0001_01_000001 t 
o application application_1434535025160_0001 
2015-06-17 11:42:44,001 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye.. 
2015-06-17 11:42:44,034 INFO org.mortbay.log: Stopped [email protected]:8042 
2015-06-17 11:42:44,035 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Applications still running : [application_14345350

我已經看到了這個錯誤

java.lang.IllegalArgumentException: Wrong FS: 
hdfs://var/log/hadoop-yarn, expected: hdfs://cdh-master:8020

在下面的帖子

：Failed to start Jobtracker and Tasktracker in CDH pseudo cluster，但這並沒有幫助我很多。

有沒有人有想法？

THX

來源

2015-06-17 delthom

更改屬性yarn.nodemanager.remote-app-log-dir在yarn-site.xml配置文件要麼：

<property> 
    <name>yarn.nodemanager.remote-app-log-dir</name> 
    <value>hdfs://cdh-master:8020/var/log/hadoop-yarn/apps</value> 
</property>

或

<property> 
    <name>yarn.nodemanager.remote-app-log-dir</name> 
    <value>/var/log/hadoop-yarn/apps</value> 
</property>

第二個選項將使用應被設置爲默認的文件系統無論如何，HDFS。

來源

2015-08-04 14:29:58 vanekjar

Hadoop紗線工作：錯誤的FS

回答

相關問題