火花SQL：無法實例org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Question

相當長的一段時間後，無法弄清楚如何運行的火花SQL二進制識別時，出現如下錯誤的根本原因：

15/12/08 14:48:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient 
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346) 
    at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:109) 
    at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:497) 
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665) 
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170) 
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193) 
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112) 
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) 
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient 
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412) 
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62) 
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72) 
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453) 
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) 
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340) 
    ... 11 more 
Caused by: java.lang.reflect.InvocationTargetException 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:422) 
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410) 
    ... 16 more 
Caused by: javax.jdo.JDOFatalUserException: Class org.datanucleus.api.jdo.JDOPersistenceManagerFactory was not found. 
NestedThrowables: 
java.lang.ClassNotFoundException: org.datanucleus.api.jdo.JDOPersistenceManagerFactory 
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1175) 
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) 
    at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) 
    at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:310) 
    at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:339) 
    at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:248) 
    at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:223) 
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73) 
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133) 
    at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58) 
    at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67) 
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:497) 
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:475) 
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:523) 
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:397) 
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356) 
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54) 
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59) 
    at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944) 
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171) 
    ... 21 more 
Caused by: java.lang.ClassNotFoundException: org.datanucleus.api.jdo.JDOPersistenceManagerFactory 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:348) 
    at javax.jdo.JDOHelper$18.run(JDOHelper.java:2018) 
    at javax.jdo.JDOHelper$18.run(JDOHelper.java:2016) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.jdo.JDOHelper.forName(JDOHelper.java:2015) 
    at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1162) 
    ... 40 more 

Answer 1

您在類路徑中缺少datanucleus-api-jdo.jar。這個jar被配置單元使用。試着弄清楚你需要這個jar的版本（可能最近會做）。你可以從這裏下載它，然後：http://repo1.maven.org/maven2/org/datanucleus/datanucleus-api-jdo/

而且通過添加以下內容到.hiverc文件添加它永久蜂巢：

add jar datanucleus-api-jdo.jar