未能在Hive中刪除數據庫

Question

我試圖在每次運行之前在配置單元中初始化數據庫。

的代碼是：

command="hive -e \"drop database if exists some_db cascade; create database some_db\""; 
eval $command; 

執行失敗，錯誤：

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to clean up java.sql.SQLSyntaxErrorException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist

它通過在重新運行，

eval $command; 

我也不能解釋時調用的事實該命令多次交替出現故障。

感謝您對Hive中發生的事情的建議，以及如何使其在第一次嘗試中發揮作用，謝謝。

---

詳情： 蜂巢2.1 AWS EMR 5.7運行時，沒有看到HIVE這種行爲1.0

完全錯誤堆棧

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Unable to clean up java.sql.SQLSyntaxErrorException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:125) at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69) at org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:259) at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:287) at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:470) at org.mariadb.jdbc.MariaDbStatement.executeUpdate(MariaDbStatement.java:486) at com.jolbox.bonecp.StatementHandle.executeUpdate(StatementHandle.java:497) at org.apache.hadoop.hive.metastore.txn.TxnHandler.cleanupRecords(TxnHandler.java:1721) at org.apache.hadoop.hive.metastore.AcidEventListener.onDropDatabase(AcidEventListener.java:51) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_database_core(HiveMetaStore.java:1098) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.drop_database(HiveMetaStore.java:1130) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:140) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:99) at com.sun.proxy.$Proxy19.drop_database(Unknown Source) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_database.getResult(ThriftHiveMetastore.java:10518) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$drop_database.getResult(ThriftHiveMetastore.java:10502) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:110) at org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:106) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:118) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: Table 'hive.COMPLETED_COMPACTIONS' doesn't exist at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:479) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.result(AbstractQueryProtocol.java:400) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:381) at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:337) at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:277) ... 27 more Blockquote

Answer 1

看起來你的蜂巢metastore模式必須升級。您可以使用schematool命令檢查模式版本。

輸出示例：

Metastore連接URL：JDBC：MySQL的：// XXXXXXXXXXXXXXXX：3306 /蜂巢createDatabaseIfNotExist =真 Metastore連接驅動程序：org.mariadb.jdbc.Driver Metastore連接用戶：蜂房 蜂房分發版本：2.1.0 Metastore架構版本：2.1.0

如果您的Metastore模式和Hive分發版本不同，請升級架構。

確保先備份，以防萬一。還要記住，可能會更改舊版Hive分發功能的模式，因此您可能無法在Hive分發版本之間共享一個Matastore。