遠程mysql使用spark的數據庫訪問錯誤

從遠程mysql數據庫訪問表，該數據庫在eclipse中本地很好地工作，但在嘗試從嘗試時拋出錯誤。遠程mysql使用spark的數據庫訪問錯誤

ERROR SparkContext: Error initializing SparkContext.java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode 
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)

低於火花提交：

spark-submit \ 
--jars /home/hduser/.ivy2/cache/com.typesafe/config/bundles/config-1.3.1.jar \ 
--class "retail.stack" --master local[2] \ 
--driver-memory 2g --executor-memory 2g --executor-cores 2 \ 
sample-spark-180417_2.11-1.0.jar \ 
/home/hduser/spark-training/workspace/demos/output/extra_in_src \ 
/home/hduser/spark-training/workspace/demos/output/extradest \ 
/home/hduser/spark-training/workspace/demos/output/bestMatch

下面的代碼：

val spark = SparkSession.builder().master("local").appName("Database") 
    .getOrCreate() 

val jdbcUsername = "root" 
val jdbcPassword = "root" 
val url = "jdbc:mysql://192.168.175.35:3306/sample?useSSL=false" 

val connectionProperties = new java.util.Properties() 
connectionProperties.put("user", jdbcUsername) 
connectionProperties.put("password", jdbcPassword) 

val querySource = "(select * from source1) as src" 
val queryDestination = "(select * from destination1) as dest" 

val source = spark.read.jdbc(url, querySource, connectionProperties).rdd.map(_.mkString(",")) 
val destination = spark.read.jdbc(url, queryDestination, connectionProperties).rdd.map(_.mkString(","))

謝謝

來源

2017-07-06 Vignesh

可以請你分享一下你試過的代碼嗎？ –

@senthilkumarp代碼更新..！ – Vignesh

https://stackoverflow.com/questions/33174386/accessing-hdfs-ha-from-spark-job-unknownhostexception-error請看這 –

我解決它通過重新配置連接字符串。它工作得很好。

來源

2017-07-07 10:52:42 Vignesh

遠程mysql使用spark的數據庫訪問錯誤

回答

相關問題