RequestHedgingRMFailoverProxyProvider的IllegalAccessError在HDP升級後在hadoop集羣中啓動Apache Twill應用程序

Question

我試圖從hadoop集羣啓動Apache Twill應用程序，集羣最近從HDP 2.2升級到HDP 2.5，但是我收到RequestHedgingRMFailoverProxyProvider的llegalAccessError類。這個類是org.apache.hadoop.yarn.client包的一部分。我在應用程序主文件中遇到這個錯誤。 「接受狀態」之後，工作狀態直接變爲「未運行狀態」。

Exception in thread "Hadoop22YarnAMClient STARTING" Exception in thread "YarnAMClientService STARTING" java.lang.IllegalAccessError: tried to access method org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object; from class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider 
at org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider.init(RequestHedgingRMFailoverProxyProvider.java:75) 
at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:163) 
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93) 
at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72) 
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:186) 
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) 
at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.startUp(Hadoop21YarnAMClient.java:77) 
at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) 
at java.lang.Thread.run(Thread.java:745) 
com.google.common.util.concurrent.ExecutionError: java.lang.IllegalAccessError: tried to access method org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object; from class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider 
at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1008) 
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) 
at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) 
at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) 
at org.apache.twill.internal.appmaster.ApplicationMasterMain$YarnAMClientService.startUp(ApplicationMasterMain.java:221) 
at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) 
at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.IllegalAccessError: tried to access method org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.getProxyInternal()Ljava/lang/Object; from class org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider 
at org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider.init(RequestHedgingRMFailoverProxyProvider.java:75) 
at org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:163) 
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93) 
at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72) 
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:186) 
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) 
at org.apache.twill.internal.yarn.Hadoop21YarnAMClient.startUp(Hadoop21YarnAMClient.java:77) 
... 2 more 

Answer 1

一般來說，當你看到IllegalAccessError這意味着你必須編譯和運行時代碼之間的運行環境不兼容。在這種情況下， getProxyInternal()方法ConfiguredRMFailoverProxyProvider現在是私有的。您需要重新編譯客戶端代碼和/或使用更新的hadoop客戶端庫連接到您的羣集。