蜂巢：DFS copyToLocal給「org.apache.hive.service.cli.HiveSQLException：錯誤在處理聲明：空」

我試圖從.hql文件做一個「copyToLocal」，像這樣：蜂巢：DFS copyToLocal給「org.apache.hive.service.cli.HiveSQLException：錯誤在處理聲明：空」

dfs -copyToLocal hdfs://nameservice1/HDFS_FOLDER1/HDFS_FOLDER2/file_name.dat /LOCAL_FOLDER1/LOCAL_FOLDER2/;

，但我得到這個下文提到的例外：

Error: Error while processing statement: null (state=,code=1) 
    org.apache.hive.service.cli.HiveSQLException: Error while processing statement: null 
      at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:239) 
      at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:225) 
      at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:244) 
      at org.apache.hive.beeline.Commands.executeInternal(Commands.java:902) 
      at org.apache.hive.beeline.Commands.execute(Commands.java:1089) 
      at org.apache.hive.beeline.Commands.sql(Commands.java:985) 
      at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1085) 
      at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:917) 
      at org.apache.hive.beeline.BeeLine.executeFile(BeeLine.java:895) 
      at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:837) 
      at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:482) 
      at org.apache.hive.beeline.BeeLine.main(BeeLine.java:465) 
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
      at java.lang.reflect.Method.invoke(Method.java:498) 
      at org.apache.hadoop.util.RunJar.run(RunJar.java:221) 
      at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 
    Caused by: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: null 
      at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:385) 
      at org.apache.hive.service.cli.operation.HiveCommandOperation.runInternal(HiveCommandOperation.java:116) 
      at org.apache.hive.service.cli.operation.Operation.run(Operation.java:327) 
      at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:424) 
      at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:401) 
      at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) 
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
      at java.lang.reflect.Method.invoke(Method.java:498) 
      at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
      at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
      at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
      at java.security.AccessController.doPrivileged(Native Method) 
      at javax.security.auth.Subject.doAs(Subject.java:422) 
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) 
      at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
      at com.sun.proxy.$Proxy21.executeStatementAsync(Unknown Source) 
      at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:258) 
      at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:500) 
      at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313) 
      at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298) 
      at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
      at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
      at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:746) 
      at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
      at java.lang.Thread.run(Thread.java:745)

但是，當我一段時間後才執行使用相同的.hql文件相同的語句，如預期其複製到本地。任何人都可以請幫我理解爲什麼會發生這種情況？

Version: Hive 1.1.0-cdh5.8.3

更新1：我用「得」，而不是「copyToLocal」，得到了相同的異常。

更新2：這個文件有粘位：

-rwxrwxrwt ..... /HDFS_FOLDER1/HDFS_FOLDER2/file_name.dat

我是userwho創建的文件，我使用DFS選項從HQL文件試圖從HDFS複製到本地文件夾。

來源

2016-12-19 Marco99

我正在使用java進行某些中間邏輯操作。我必須獲得

org.apache.hadoop.fs.FileSystem

對象，並做了

fileSystem.copyToLocalFile(new Path("hdfs://nameservice1/HDFS_FOLDER1/HDFS_FOLDER2/file_name.dat"), new Path(localDir));

來源

2016-12-20 10:03:58 Marco99

HADOOP命令複製到本地文件位置。

hdfs dfs -copyToLocal <input> <output>

這是您的具體例子：

hdfs dfs -copyToLocal /nameservice1/HDFS_FOLDER1/HDFS_FOLDER2/file_name.dat /LOCAL_FOLDER1/LOCAL_FOLDER2;

這是過時的版本：

hadoop fs -copyToLocal /nameservice1/HDFS_FOLDER1/HDFS_FOLDER2/file_name.dat /LOCAL_FOLDER1/LOCAL_FOLDER2;

HIVE COMMAND拷貝文件到本地文件的位置。

INSERT OVERWRITE LOCAL DIRECTORY '/your/directory/in/local' SELECT * FROM myTable

來源

2016-12-19 14:17:01 invoketheshell

它沒有工作。 – Marco99

我試圖運行該命令作爲HQL文件的一部分。您提到的命令的棄用版本需要從shell腳本運行，對吧？ – Marco99

我添加了配置單元代碼。 – invoketheshell

蜂巢：DFS copyToLocal給「org.apache.hive.service.cli.HiveSQLException：錯誤在處理聲明：空」

回答

相關問題