2014-10-28 70 views
1

我用過spark SQL connect hive,基本查詢還行,但是無法創建索引,會不會引發SQL支持索引?Spark SQL(版本1.1.0)是否支持配置單元索引?

DROP TABLE test_tabe; 
CREATE TABLE test_tabe 
(
.... 
,area STRING 
... 
) 
ROW FORMAT DELIMITED 
FIELDS TERMINATED BY '\t' 
LINES TERMINATED BY '\n' 
STORED AS RCFILE 
--STORED AS SEQUENCEFILE 
; 
CREATE INDEX idx_compact_index ON TABLE test_tabe (area) AS 'COMPACT' WITH DEFERRED REBUILD 

#!!!!Execute always,no error,spark web page no corresponding job !!!! 
ALTER INDEX idx_compact_index ON test_tabe REBUILD; 

insert overwrite table test.test_tabe select * from test.new_mkt_browse_label_10m 

謝謝。

回答

1

spark中不支持Hive索引。它們並不重要,因爲火花在內存計算中。 你有任何機會運行索引配置單元查詢與類似的火花查詢之間的比較? 您可以檢出更多關於索引未被執行的信息here

+0

Hive vs Spark SQL vs Impala http://blog.atscale.com/how-different-sql-on-hadoop-engines-satisfy-bi-workloads – 2016-07-27 16:55:39

相關問題