2016-10-17 53 views
1

我使用spark-submit將我的jar與依賴關係提交給了spark。在我的罐子的主要方法我想創建HttpAsyncCliens實例並執行有一定的要求(的Apache HTTP異步客戶端庫):spark中的NoSuchMethodError提交

val httpClient = HttpAsyncClients.custom.setMaxConnTotal(10).setMaxConnPerRoute(10).build 
httpClient.start() 
httpClient.execute(new HttpGet("https://google.com"), new FutureCallback[HttpResponse] { 
    /* callbacks */ 
}) 

它拋出異常:

Exception in thread "pool-1-thread-1" java.lang.NoSuchMethodError: org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:315) at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:191) at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) at java.lang.Thread.run(Thread.java:745) Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90) at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:74) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:107) at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:91) at spark.Application$.main(Application.scala:37) at spark.Application.main(Application.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731) at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

好像沒有基於HTTP在我的罐子核心的依賴,但我之前或HTTP客戶端創建和請求後調用代碼這個方法(org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V):

org.apache.http.util.Asserts.check(true, "test", "test2") // it produces no exception 

val httpClient = HttpAsyncClients.custom.setMaxConnTotal(10).setMaxConnPerRoute(10).build 
httpClient.start() 
httpClient.execute(new HttpGet("https://google.com"), new FutureCallback[HttpResponse] { 
    /* callbacks */ 
}) // it produces exception 

爲什麼我NoSuchMethodErr或者如果我可以從代碼中的相同類路徑調用此方法?

阿帕奇httpasyncclient V4.1

回答

1

這裏的解決方案是與版本httpcomponents.httpcore的是在火花的即時類路徑同步。版本1.6版本是4.0.1。

你將不得不解壓縮火花瓶來查看META-INF等。經過一些偵探工作後,事情會好起來的!

相關問題