錯誤使用SPARK時：java.lang.NoClassDefFoundError：調度/ HTTP

Question

使用時總是火花提交此錯誤發生：

ERROR Executor: Exception in task 0.0 in stage 137.0 (TID 35) 
java.lang.NoClassDefFoundError: dispatch/Http$ 

雖然我有進口的lib（我使用的Scala）：

import dispatch._, Defaults._ 

的代碼部分：

def publishToNsq(data: String) = { 
     val topic = "post_similar_items_to_elasticsearch" 
     val host = "192.168.100.160:4151" 
     def myRequest = url(s"$host/pub?topic=$topic") 
     def myRequestAsJson = myRequest.setContentType("application/json", "UTF-8") 
     def myPostWithBody = myRequestAsJson << data 
     val response = Http(myPostWithBody OK as.String) 
     println(response) 
    } 

這是我build.sbt的一部分：

scalaVersion := "2.10.5" 

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.6.1" 
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.6.1" 
libraryDependencies += "org.json4s" %% "json4s-native" % "3.4.0" 
libraryDependencies += "org.json4s" %% "json4s-jackson" % "3.4.0" 
libraryDependencies += "org.scalaj" %% "scalaj-http" % "1.1.4" 
libraryDependencies += "net.databinder.dispatch" %% "dispatch-core" % "0.11.2" 
resolvers += Resolver.mavenLocal 

任何解決方案？

Answer 1

的依賴是不是在罐子裏，它會被解決，如果你配置爲在java.lang.NoClassDefFoundError: org/apache/spark/streaming/twitter/TwitterUtils