當我這樣做SBT包或SBT組件我得到錯誤等:apache的火花predictProbabilities不是會員NaiveBayesModel
值predictProbabilities不是org.apache.spark.mllib.classification.NaiveBayesModel
成員,但相同的代碼工作在我的火花殼
/home/user1/spark-1.5.2/bin/spark-shell --packages com.databricks:火花csv_2.10:1.4.0
我使用spark-1.5.2和scala 2.10.4 sbt 0.13.8
我使用centos 6.5 linux
我確實在此鏈接中看到預測概率。 https://spark.apache.org/docs/1.5.2/api/scala/index.html#org.apache.spark.mllib.classification.NaiveBayesModel
我不知道是否需要sbt程序集或io.prediction 但我試過了。沒什麼區別。獲取相同的錯誤。
有沒有人碰到這樣的問題。 如果是這樣,你可以讓我知道解決方案。
謝謝!
我的代碼是
======================================= ===================
import org.apache.spark.mllib.classification.{NaiveBayes, NaiveBayesModel}
import org.apache.spark.mllib.util.MLUtils
val trainData = MLUtils.loadLibSVMFile(sc, "file:///home/user1/nbtest/trainData.nb.svm")
val testData = MLUtils.loadLibSVMFile(sc, "file:///home/user1/nbtest/testData.nb.svm")
val model = NaiveBayes.train(trainData, lambda = 1.0, modelType = "multinomial")
val predictionAndLabel = testData.map(p => (model.predict(p.features), p.label))
val accuracy = 1.0 * predictionAndLabel.filter(x => x._1 == x._2).count()/testData.count()
val testDataFeatures = testData.map(p => p.features)
val probabilities = model.predictProbabilities (testDataFeatures)
======================== ===========================================
我的/ home /user1/nbtest/build.sbt是這樣
=================================== ==========================
name := "testBayes"
version := "1.0"
scalaVersion := "2.10.4"
scalaSource in Compile := baseDirectory.value/""
libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.2"
libraryDependencies += "org.apache.spark" % "spark-sql_2.10" % "1.5.2"
libraryDependencies += "org.apache.spark" % "spark-mllib_2.10" % "1.4.0"
libraryDependencies += "com.databricks" % "spark-csv_2.10" % "1.4.0"
libraryDependencies += "io.prediction" % "core_2.10" % "0.9.6"
libraryDependencies += "io.prediction" % "sdk" % "0.5.0"
assemblyMergeStrategy in assembly := {
case PathList("META-INF", xs @ _*) => MergeStrategy.discard
case x => MergeStrategy.first
============================================ =================
我已經在/ home /用戶1/nbtest /項目 assembly.sbt plugins.sbt
assembly.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.12.0")
plugins.sbt
logLevel := Level.Warn
================================================ =============
你願意給我解釋,你認爲你的sbt文件呢? – eliasah
我做sbt包來構建我使用spark-submit運行的目標jar文件。 – marrel
我的問題是它是做什麼的?我知道什麼是sbt。我只是想幫你弄清楚你的問題。 – eliasah