0
我嘗試將spark數據框的內容寫入mysql表。但我總是得到以下錯誤:將spark數據框寫入mysql表時出錯
java.sql.SQLException: No suitable driver found for 'URL'
我正在使用spark-1.6.0。對數據庫的查詢沒有任何問題: 獲取設置權限後,我可以運行查詢並獲得結果,而不會有任何問題。
val mysqlDriver = "Driver"
val mysqlUsername = "name"
val mysqlPwd = "pwd"
val mysqlUrl = "jdbc:mysql://myUrl"
val sqlContext = new org.apache.spark.sql.SQLContext(sc)
Class.forName(mysqlDriver).newInstance
val dataRDD = new JdbcRDD(sc,() =>
DriverManager.getConnection(mysqlUrl, mysqlUsername, mysqlPwd) ,
"a query",
0,
1000000,
2,
r => r.getString("something") + ", " + r.getString("somethingmore") + ", " + r.getString("somethingelse"))
然而,寫一個數據幀到表不起作用:
todb.insertIntoJDBC(mysqlUrl + "/db" + "?user=name&password=pwd", "table", false)
因爲這種方法似乎過時了,我也試過:
val prop = new java.util.Properties
prop.setProperty("user", mysqlUsername)
prop.setProperty("password",mysqlPwd)
試過這兩個版本:
prop.setProperty("driver", "/somedir/mysql-connector-java-5.1.37-bin.jar")
prop.setProperty("driver", "mysqlDriver")
和這兩個版本:
todb.write.jdbc(mysqlUrl + "/boosting" + "?user=mysqlUsername&password=mysqlPwd", "boosting.user_cluster", prop)
todb.write.jdbc(mysqlUrl, "table", prop)
我也開始了火花shell時把駕駛員的水瓶中的–jars
論據之一。此外,我試圖設置此:
val conf = new SparkConf().setJars(Array("somejar.jar", "/somedir/mysql-connector-java-5.1.37-bin.jar"))
任何有關這個問題的幫助將不勝感激!