2016-08-16 21 views
4

我的代碼工作很好火花外殼:IDEA爲什麼找不到toDS()和toDF()函數?

scala> case class Person(name:String,age:Int) 
defined class Person 

scala> val person = Seq(Person("ppopo",23)).toDS() 
person: org.apache.spark.sql.Dataset[Person] = [name: string, age: int] 

scala> person.show() 
+-----+---+ 
| name|age| 
+-----+---+ 
|ppopo| 23| 
+-----+---+ 

,但錯在IDEA:

enter image description here
我已經進口的所有罐子在「火花2.0.0彬hadoop2.7 /罐子/「,但仍然無法找到這個功能。

回答

7

我發現這個問題,usering TODS()之前添加一個依賴:

val ss = SparkSession.builder().appName("DataSet Test") 
    .master("local[*]").getOrCreate() 

// This import is needed 
import ss.implicits._ 
val simpleDS = Seq(Person("po",12)).toDS() 
simpleDS.show()