2015-04-06 15 views
2

我試圖執行此bash的代碼時,有一個問題中運行的黑斑羚殼:有關如何在shell腳本

function createImpalaPartition() { 

period_id=$1; 
database=$2 
node=$3 

actual_full=$(date [email protected]"$period_id" +%Y/%m/%d/%H/%M/) 
template="use c2d;create EXTERNAL TABLE exptopology_$period_id (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/$actual_full'" 

echo "template is $template"; 
#impala-shell -V -i $node -d $database -q $template 
impala-shell -V -i $node -q $template 
} 

這是我如何調用它:

createImpalaPartition $actual $impalaDatabase $impalaNode 

其中

actual=$(date +'%s') 
impalaDatabase="dbName" 
impalaNode="name_of_the_node" 

該腳本的執行返回:

[[email protected] ~]$ createImpalaPartition $actual $impalaDatabase $impalaNode 
template is use c2d;create EXTERNAL TABLE exptopology_1428326587 (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/2015/04/06/14/23/' 
Error, could not parse arguments "c2d;create EXTERNAL TABLE exptopology_1428326587 (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/2015/04/06/14/23/'" 
Usage: impala_shell.py [options] 

正如你所看到的,我必須使用shell腳本創建表。

UPDATE:

下面這個link,我可以看到黑斑羚殼可以用這種方式,但我沒有使用正確的PARAMS。

我已經使用了-f而不是-q,同樣的錯誤依然存在。有人可以幫我嗎?

回答

2

你需要引用的$template擴大爲整個字符串作爲單個參數進行處理,以impala-shell

impala-shell -V -i $node "$template" 

其中-V啓用詳細輸出。對於非詳細(安靜)輸出,用-q替換-V

0

最後我發現如何解決我的問題。

function createImpalaPartition() { 

period_id=$1; 
database=$2 
node=$3 

actual_full=$(date [email protected]"$period_id" +%Y/%m/%d/%H/%M/) 
#UC=$(impala-shell -r -q "select count(1) from table where condition=1" -d $DB -i $HOST -B) 
# attention, i have to use this way impala-shell 
impala-shell -V -i $node -d $database -q "create EXTERNAL TABLE exptopology_$period_id (child_id bigint,parent_id bigint,level INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' WITH SERDEPROPERTIES ('serialization.format'=',', 'field.delim'=',') STORED AS TEXTFILE LOCATION '/hfc/sip/service/topology/$actual_full'" 
} 

我不能使用模板var與create命令,我必須以這種方式傳遞命令。

+0

使用'-q'您不能傳遞USE語句和另一個查詢,完全限定默認數據庫外部任何表的名稱。 (或者使用-f選項來傳遞帶有USE語句的文件,然後是其他查詢。)[請參閱(http://www.cloudera.com/documentation/archive/impala/2-x/2-0-x) /topics/impala_shell_options.html) – 2016-05-12 16:47:19