2017-08-09 153 views
0

我的平臺:CDH 5.10快速入門VM和Teradata TDEXPRESS 16 VM版本。 我正在使用Sqoop和Teradata JDBC驅動程序- tdgssconfig.jar,teradata-connector-1.5.3.jar,terajdbc4.jar 我的sqoop作業失敗,出現以下錯誤。Teradata的Sqoop incremantal導入失敗

有什麼建議嗎?

[[email protected] sqoop]$ sqoop job --create incjobtd1 -- import --connect jdbc:teradata://192.168.142.129/DATABASE=teradb --driver com.teradata.jdbc.TeraDriver --username dbc --password dbc --table ACCT --incremental lastmodified 
     --check-column ECF_XTRC_TS  --target-dir /user/cloudera/teradb/acct2 -m 1 --as-parquetfile --append 
    Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. 
    Please set $ACCUMULO_HOME to the root of your Accumulo installation. 
    17/08/09 01:44:49 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.10.0 
    17/08/09 01:44:54 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 
    [[email protected] sqoop]$ sqoop job --exec incjobtd1 
    Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail. 
    Please set $ACCUMULO_HOME to the root of your Accumulo installation. 
    17/08/09 01:45:14 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.10.0 
    Enter password: 
    17/08/09 01:45:25 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 
    17/08/09 01:45:25 INFO manager.SqlManager: Using default fetchSize of 1000 
    17/08/09 01:45:25 INFO tool.CodeGenTool: Beginning code generation 
    17/08/09 01:45:25 INFO tool.CodeGenTool: Will generate java class as codegen_ACCT 
    17/08/09 01:45:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM ACCT AS t WHERE 1=0 
    17/08/09 01:45:32 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM ACCT AS t WHERE 1=0 
    17/08/09 01:45:32 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/lib/hadoop-mapreduce 
    Note: /tmp/sqoop-cloudera/compile/fc68a3f34abd80bde713cf1709dbc330/codegen_ACCT.java uses or overrides a deprecated API. 
    Note: Recompile with -Xlint:deprecation for details. 
    17/08/09 01:45:54 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-cloudera/compile/fc68a3f34abd80bde713cf1709dbc330/codegen_ACCT.jar 
    17/08/09 01:46:06 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM ACCT AS t WHERE 1=0 
    17/08/09 01:46:07 ERROR manager.SqlManager: SQL exception accessing current timestamp: java.sql.SQLException: [Teradata Database] [TeraJDBC 16.00.00.23] [Error 3706] [SQLState 42000] Syntax error: expected something between '(' and ')'. 
    java.sql.SQLException: [Teradata Database] [TeraJDBC 16.00.00.23] [Error 3706] [SQLState 42000] Syntax error: expected something between '(' and ')'. 
      at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:309) 
      at com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:103) 
      at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:311) 
      at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(StatementReceiveState.java:200) 
      at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:137) 
      at com.teradata.jdbc.jdbc_4.statemachine.StatementController.run(StatementController.java:128) 
      at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:389) 
      at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:331) 
      at com.teradata.jdbc.jdbc_4.TDStatement.doNonPrepExecuteQuery(TDStatement.java:319) 
      at com.teradata.jdbc.jdbc_4.TDStatement.executeQuery(TDStatement.java:1121) 
      at org.apache.sqoop.manager.SqlManager.getCurrentDbTimestamp(SqlManager.java:987) 
      at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:328) 
      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:498) 
      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) 
      at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:213) 
      at org.apache.sqoop.tool.JobTool.run(JobTool.java:268) 
      at org.apache.sqoop.Sqoop.run(Sqoop.java:143) 
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) 
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) 
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) 
      at org.apache.sqoop.Sqoop.main(Sqoop.java:236) 
    17/08/09 01:46:07 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Could not get current time from database 
      at org.apache.sqoop.tool.ImportTool.initIncrementalConstraints(ImportTool.java:330) 
      at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:498) 
      at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) 
      at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:213) 
      at org.apache.sqoop.tool.JobTool.run(JobTool.java:268) 
      at org.apache.sqoop.Sqoop.run(Sqoop.java:143) 
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) 
      at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) 
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) 
      at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) 
      at org.apache.sqoop.Sqoop.main(Sqoop.java:236) 

    [[email protected] sqoop]$ 

回答

0

您已使用--appendlastmodified一併修改。這是不正確的。請在下面找到它們之間的詳細區別。

Sqoop支持兩種類型的增量導入:append和lastmodified。您可以使用--incremental參數來指定要執行的增量導入的類型。

您應該在導入一個表格的時候指定append模式,在這個表格中,隨着行ID值的增加不斷增加新行。用--check-column指定包含該行ID的列。 Sqoop導入檢查列的值大於--last-value指定值的行。

Sqoop支持的備用表更新策略稱爲lastmodified模式。當源表的行可能被更新時,您應該使用它,每個這樣的更新都會將最後修改的列的值設置爲當前的時間戳。

所以假設你的表格有一些新的插入和sone網絡更新以及你的Sqoop作業。

sqoop job --create incjobtd1 -- import --connect jdbc:teradata://192.168.142.129/DATABASE=teradb \ --driver com.teradata.jdbc.TeraDriver --username dbc --password dbc \ --table ACCT --incremental lastmodified --check-column ECF_XTRC_TS \ --target-dir /user/cloudera/teradb/acct2 -m 1 --as-parquetfile

+0

這是一個錯誤。相同的聲明已經過測試,沒有錯誤。 – Rana