ClassNotFoundException使用java客戶端的hive + hbase集成

我有一個hive + hbase集成集羣。ClassNotFoundException使用java客戶端的hive + hbase集成

當我嘗試通過配置單元的java客戶端執行查詢時，有時會發生 a ClassNotFoundException。

我的Java代碼：

final Connection conn = DriverManager.getConnection(URL); 
final ResultSet rs = conn.executeQuery("SELECT count(*) FROM test_table WHERE (source = '0' AND ur_createtime BETWEEN '20121031000000' AND '20121031235959')");

我可以執行SQL：在蜂房CLI國防部SELECT count(*) FROM test_table WHERE (source = '0' AND ur_createtime BETWEEN '20121031000000' AND '20121031235959')，並獲得查詢結果，所以在我的SQL沒有錯誤。

客戶端異常：

Caused by: java.sql.SQLException: Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask 
    at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:189) 
... 23 more

服務器端異常（Hadoop的JobTracker的）：

2012-11-05 18:55:39,443 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201210301133_0112_m_000000_3: java.io.IOException: Cannot create an instance of InputSplit class = org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit 
    at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:146) 
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67) 
    at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40) 
    at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:396) 
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:412) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) 
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Unknown Source) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) 
    at org.apache.hadoop.mapred.Child.main(Child.java:249) 
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseSplit 
    at java.net.URLClassLoader$1.run(Unknown Source) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(Unknown Source) 
    at java.lang.ClassLoader.loadClass(Unknown Source) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) 
    at java.lang.ClassLoader.loadClass(Unknown Source) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Unknown Source) 
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819) 
    at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:143) 
    ... 10 more

我hive-env.sh

export HIVE_AUX_JARS_PATH=/data/install/hive-0.9.0/lib/hive-hbase-handler-0.9.0.jar,/data/install/hive-0.9.0/lib/hbase-0.92.0.jar,/data/install/hive-0.9.0/lib/zookeeper-3.4.2.jar

我的蜂箱現場.xml

<property> 
    <name>hive.zookeeper.quorum</name> 
    <value>hadoop01,hadoop02,hadoop03</value> 
    <description>The list of zookeeper servers to talk to. This is only needed for read/write locks.</description> 
</property>

我做起如下節儉服務：

hive --service hiveserver -p 10000 &

服務器端錯誤日誌中說，HBaseSplit沒有找到。但爲什麼？我怎樣才能解決這個問題？

來源

2012-11-05 scarcer

此問題的解決方法是要麼你可以複製的jar文件蜂房HBase的處理程序-0.9.0-cdh4.1.2，HBase的-0.92.1-cdh4.1.2安全等，以Hadoop的lib文件夾或將這些路徑添加到HADOOP_CLASSPATH環境變量中。

來源

2013-01-23 13:14:18

創建$ HIVE_HOME一個文件夾auxlib，並把所有蜂房HBase的處理程序，HBase的罐子到該文件夾

添加以下行至$ HIVE_HOME/conf目錄/蜂房的site.xml

<property> 
<name>hive.aux.jars.path</name> 
<value>file:///<absolute-path-of-all-auxlib-jars></value> 
</property>

重啓蜂巢服務器

來源

2013-03-17 20:31:28 frisky

如果您沒有訪問配置文件，您可以在dd將jar添加到使用--auxpath開關的配置單元類路徑中：

hive --auxpath /path/to/hive-hbase-handler-0.10.0-cdh4.2.0.jar,/path/to/hbase.jar

來源

2013-10-09 14:56:39

ClassNotFoundException使用java客戶端的hive + hbase集成

回答

相關問題