Sqoop增量導入失敗

我有下面的表格：mergetab在MYSQL中。Sqoop增量導入失敗

id | name  | city  | rectime    | 
---+-----------+-------------+---------------------+ 
1 | Sidhartha | Hyderabad | 2017-04-18 15:31:22 | 
2 | Saketh | Bengaluru | 2017-04-18 15:32:37 | 
3 | Sunny  | Mumbai  | 2017-04-18 15:32:57 | 
4 | Bobby  | Delhi  | 2017-04-18 15:33:15 |

我已經在裏面插入一條記錄，它看起來如下：

id | name  | city  | rectime    | 
---+-----------+-------------+---------------------+ 
1 | Sidhartha | Hyderabad | 2017-04-18 15:31:22 | 
2 | Saketh | Bengaluru | 2017-04-18 15:32:37 | 
3 | Sunny  | Mumbai  | 2017-04-18 15:32:57 | 
4 | Bobby  | Delhi  | 2017-04-18 15:33:15 | 
5 | Madhavi | Dharmavaram | 2017-04-18 16:57:09 |

我試着用下面的命令做了sqoop增量導入：

sqoop import --connect jdbc:mysql://127.0.0.1/mydb --table mergetab --username root --password cloudera --hive-import --hive-table mergetab --incremental-append --check-column id --last-value $(hive -S -e "select max(id) from mergetab");

即時得到的以下錯誤，我無法理解我的sqoop命令有什麼問題。錯誤：

17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Error parsing arguments for import: 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: WARN: 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: The 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: method 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: class 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: org.apache.commons.logging.impl.SLF4JLogFactory#release() 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: was 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: invoked. 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: WARN: 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: Please 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: see 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: http://www.slf4j.org/codes.html#release 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: for 
17/04/18 17:35:26 ERROR tool.BaseSqoopTool: Unrecognized argument: an

我在Cloudera VM中練習這個。任何人都可以讓我在這裏做什麼錯誤？

來源

2017-04-18 Sidhartha

你只是試圖運行'蜂巢-S -e」的值.. .wtf ...「命令在as中地獄，看看它產生了什麼樣的垃圾？ –

我試過了。 FAILED：ParseException行1：0無法識別'hive'附近的輸入 - ''S'。但即使沒有'-S'，我仍然收到相同的錯誤信息。 – Sidhartha

與您的代碼行--last-value $(hive -S -e "select max(id) from mergetab")的問題是，蜂巢總會打印頭/即使在靜音模式下記錄信息，所以最後的值總是會得到價值一樣 -

SLF4J: Class path contains multiple SLF4J bindings. 
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hive/2.1.0/libexec/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hadoop/2.7.3/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] 
5

你需要或者寫一些扭曲shell腳本提取的最後一行，也可以使用直線與showHeader和outputformat如下圖所示先手直接分配給變量

--last value $(beeline --showHeader=false --outputformat=tsv2 -e "your query")

來源

2017-04-21 02:54:11 Pushkr

我給了最後一個值：--last-value $（beeline --showHeader = false --outputformat = tsv2 -e「從mergetab中選擇max（id）」）進入錯誤信息：請設置$ ACCUMULO_HOME到您的Accumulo安裝的根目錄。 17/04/21 12:40:22信息sqoop.Sqoop：運行Sqoop版本：1.4.6-cdh5.8.0 選項丟失的參數：上一個值 – Sidhartha

這是警告我想。爲了解決這個問題，你需要設置$ ACCUMULO_HOME – Pushkr

Sqoop增量導入失敗

回答

相關問題