連接到Spark Thrift Server的ODBC配置

這個問題看起來可能重複，事實上，我已經看到了一些與此相關的問題，但並不完全具有相同的錯誤，所以我要求查看是否有人有線索。連接到Spark Thrift Server的ODBC配置

我已經設置了一個使用默認設置運行的Spark Thrift Server。 Spark版本是2.1，它運行在YARN上（Hadoop 2.7.3）

事實是，我無法設置Simba配置單元ODBC驅動程序和Microsoft，因此ODBC設置中的測試成功。

這是我使用微軟蜂房ODBC驅動程序的配置：

當我打的測試按鈕，顯示的錯誤消息如下：

在Spark Thrift Server日誌中顯示以下內容：

17/09/15 17:31:36 INFO ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V1 
17/09/15 17:31:36 INFO SessionState: Created local directory: /tmp/00abf145-2928-4995-81f2-fea578280c42_resources 
17/09/15 17:31:36 INFO SessionState: Created HDFS directory: /tmp/hive/test/00abf145-2928-4995-81f2-fea578280c42 
17/09/15 17:31:36 INFO SessionState: Created local directory: /tmp/vagrant/00abf145-2928-4995-81f2-fea578280c42 
17/09/15 17:31:36 INFO SessionState: Created HDFS directory: /tmp/hive/test/00abf145-2928-4995-81f2-fea578280c42/_tmp_space.db 
17/09/15 17:31:36 INFO HiveSessionImpl: Operation log session directory is created: /tmp/vagrant/operation_logs/00abf145-2928-4995-81f2-fea578280c42 
17/09/15 17:31:36 INFO SparkExecuteStatementOperation: Running query 'set -v' with 82d7f9a6-f2a6-4ebd-93bb-5c8da1611f84 
17/09/15 17:31:36 INFO SparkSqlParser: Parsing command: set -v 
17/09/15 17:31:36 INFO SparkExecuteStatementOperation: Result Schema: StructType(StructField(key,StringType,false), StructField(value,StringType,false), StructField(meaning,StringType,false))

如果我連接的直線（該工程確定）來使用JDBC驅動程序，這些都是日誌：

17/09/15 17:04:24 INFO ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8 
17/09/15 17:04:24 INFO SessionState: Created HDFS directory: /tmp/hive/test 
17/09/15 17:04:24 INFO SessionState: Created local directory: /tmp/c0681d6f-cc0f-40ae-970d-e3ea366aa414_resources 
17/09/15 17:04:24 INFO SessionState: Created HDFS directory: /tmp/hive/test/c0681d6f-cc0f-40ae-970d-e3ea366aa414 
17/09/15 17:04:24 INFO SessionState: Created local directory: /tmp/vagrant/c0681d6f-cc0f-40ae-970d-e3ea366aa414 
17/09/15 17:04:24 INFO SessionState: Created HDFS directory: /tmp/hive/test/c0681d6f-cc0f-40ae-970d-e3ea366aa414/_tmp_space.db 
17/09/15 17:04:24 INFO HiveSessionImpl: Operation log session directory is created: /tmp/vagrant/operation_logs/c0681d6f-cc0f-40ae-970d-e3ea366aa414 
17/09/15 17:04:24 INFO SparkSqlParser: Parsing command: use default 
17/09/15 17:04:25 INFO HiveMetaStore: 1: get_database: default 
17/09/15 17:04:25 INFO audit: ugi=vagrant ip=unknown-ip-addr cmd=get_database: default 
17/09/15 17:04:25 INFO HiveMetaStore: 1: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 
17/09/15 17:04:25 INFO ObjectStore: ObjectStore, initialize called 
17/09/15 17:04:25 INFO Query: Reading in results for query "[email protected]" since the connection used is closing 
17/09/15 17:04:25 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 
17/09/15 17:04:25 INFO ObjectStore: Initialized ObjectStore

來源

2017-09-15 Daniel Martínez

嗯，我設法通過安裝而不是蜂巢一個微軟的Spark ODBC驅動程序連接成功。看起來問題與驅動程序在發現它不是基於某些服務器屬性的Hive2服務器時拒絕連接到Spark Thrift Server的問題有關。我懷疑Hive2和Spark節點服務器之間的線路層存在實際差異，因爲後者是協議層（Thrift）沒有改變的前者的端口，但是無論如何，解決方案是轉移到該驅動程序並進行配置它與Hive2一樣：

Microsoft® Spark ODBC Driver

來源

2017-09-16 08:48:32

連接到Spark Thrift Server的ODBC配置

回答

相關問題