使用Scala TTL選項保存到Cassandra的火花

Question

我在Scala中使用Cassandra Spark連接器。 這是我的示例代碼，它正在將數據保存到Cassandra。

val data = rdd.map 
(f => new CassandraRow(IndexedSeq("pk", "count"), IndexedSeq(f._1, f._2.toString()))) 

data.saveToCassandra("keyspace", "table") 

然後，我嘗試使用TTL。這裏是我添加TTL的示例代碼。

增加進口

import com.datastax.spark.connector.writer.{TTLOption, WriteConf} 

和增加的TTL saveToCassandra

data.saveToCassandra 
("keyspace", "table", writeConf = WriteConf(ttl = TTLOption.constant(604800)) 

然而，當我編譯它，它就會錯誤。

錯誤消息

[error] bad symbolic reference. A signature in TTLOption.class refers to term streaming 
[error] in package org.apache.spark which is not available. 
[error] It may be completely missing from the current classpath, or the version on 
[error] the classpath might be incompatible with the version used when compiling TTLOption.class. 
[error] one error found 
[error] (compile:compile) Compilation failed 
[error] Total time: 4 s, completed Mar 31, 2015 11:55:14 AM 

build.sbt

name := "Cassandra" 

version := "1.0" 

scalaVersion := "2.10.4" 

libraryDependencies += "org.apache.spark" % "spark-core_2.10" % "1.2.1" 

libraryDependencies += "org.apache.spark" % "spark-sql_2.10" % "1.2.1" 

libraryDependencies += "com.datastax.spark" % "spark-cassandra-connector_2.10" % "1.2.0-rc2" 

有我的代碼的問題？

Answer 1

這是當前連接器中有關依賴於流式火花庫的已知錯誤。

https://datastax-oss.atlassian.net/browse/SPARKC-113

如果這是阻塞，你需要一種解決方法，你可以只包括火花流庫作爲構建文件的依賴性。

libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "1.2.1"