2012-09-03 42 views
4

我試圖運行分佈式殼爲例(使用Hadoop的SVN結帳,這就是爲什麼版本爲3.0.0-SNAPSHOT):的Hadoop /紗線分佈殼例如

yarn jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar \ 
-jar share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar \ 
org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command whoami 

然而這是行不通的:

12/09/03 13:44:37 FATAL distributedshell.Client: Error running CLient 
java.lang.reflect.UndeclaredThrowableException 
at org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:128) 
at org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getClusterMetrics(ClientRMProtocolPBClientImpl.java:123) 
at org.hadoop.yarn.client.YarnClientImpl.getYarnClusterMetrics(YarnClientImpl.java:163) 
at org.apache.hadoop.yarn.applications.distributedshell.Client.run(Client.java:316) 
at org.apache.hadoop.yarn.applications.distributedshell.Client.main(Client.java:164) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:601) 
at org.apache.hadoop.util.RunJar.main(RunJar.java:208) 
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB 
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.getProtocolImpl(ProtobufRpcEngine.java:398) 
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:456) 
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:898) 
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1732) 
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1728) 
at java.security.AccessController.doPrivileged(Native Method) 
at javax.security.auth.Subject.doAs(Subject.java:415) 
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367) 
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1726) 

at org.apache.hadoop.ipc.Client.call(Client.java:1164) 
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202) 
at $Proxy7.getClusterMetrics(Unknown Source) 
at org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getClusterMetrics(ClientRMProtocolPBClientImpl.java:121) 
... 8 more 

根本的問題似乎是在第二跡:

Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB 

有誰知道Hadoops ProtoBufRPC的協議註冊是如何工作的?任何想法如何調試?

編輯:隨着Hadoop版本2.0.1-alpha,它的工作稍好。

12/09/03 18:43:14 INFO distributedshell.Client: Application did not finish. YarnState=FAILED, DSFinalStatus=FAILED. Breaking monitoring loop 
12/09/03 18:43:14 ERROR distributedshell.Client: Application failed to complete successfully 

因此,也許我的構建沒有正常工作。任何關於導致上述問題的想法(我真的很喜歡使用HEAD,因爲我打算在MapReduce之外做一些低級別的實驗)?或HEAD部分損壞,HEAD上的分佈式shell是否適合您?

我自己的(沒有工作......)客戶端仍然無法與同樣的錯誤:

Caused by: java.io.IOException: Unknown protocol: org.apache.hadoop.yarn.api.ClientRMProtocolPB 

回答

3

原來我自己代碼的主要問題是,我天真地實例化的Configuration類而不是實例化YarnConfiguration。這樣,紗線配置文件沒有被讀取,它試圖聯繫服務器的默認端口 - 這不符合我的設置。

distributedshell示例中似乎存在相同的錯誤。