在sqlite中內部連接的查詢優化

Question

我有一個包含3個表的masterInfo，primDescT，secDescT數據庫。

CREATE TABLE masterInfo (id INTEGER PRIMARY KEY AUTOINCREMENT, 
primDescId INTEGER, 
secDescId INTEGER, 
category INTEGER, 
UNIQUE(primDescId, secDescId, category)); 

CREATE TABLE primDescT (id INTEGER PRIMARY KEY, 
primDesc nvarchar(512)); 

CREATE TABLE secDescT (id INTEGER PRIMARY KEY, 
secDesc nvarchar(512)); 

INSERT INTO primDescT VALUES(1,'XXXX'); 
INSERT INTO primDescT VALUES(2,'YYYY'); 
INSERT INTO primDescT VALUES(3,'ZZZZ'); 
INSERT INTO primDescT VALUES(4,'SSSS'); 

INSERT INTO secDescT VALUES(1,'AAA'); 
INSERT INTO secDescT VALUES(2,'BBB'); 
INSERT INTO secDescT VALUES(3,'CCC'); 

INSERT INTO masterInfo VALUES(1,1,1,1); 
INSERT INTO masterInfo VALUES(2,2,2,2); 
INSERT INTO masterInfo VALUES(3,3,1,1); 
INSERT INTO masterInfo VALUES(4,4,3,2); 

表，masterInfo在primDescT中有1765137行，312210行，在secDescT中有105458行。

我已經使用下面的查詢來獲取結果。

SELECT m.id AS pId, 
primDesc AS pDescr, secDesc AS sDescr, category AS category 
FROM masterInfo m 
INNER JOIN primDescT ON primDescT.id = m.primDescId 
INNER JOIN secDescT ON secDescT.id = m.secDescId 
WHERE m.category IN ('1','2') ORDER BY pDescr ASC LIMIT 100 OFFSET 0 

上面的查詢需要8秒的響應時間。

但是，如果我設置偏移量爲1756300，那麼它將需要53秒。

SELECT m.id AS pId, 
primDesc AS pDescr, secDesc AS sDescr, category AS category 
FROM masterInfo m 
INNER JOIN primDescT ON primDescT.id = m.primDescId 
INNER JOIN secDescT ON secDescT.id = m.secDescId 
WHERE m.category IN ('1','2') ORDER BY pDescr ASC LIMIT 100 OFFSET 1756300 

如何優化上述查詢以在3秒內獲取？

Answer 1

這些查詢的問題是ORDER BY：在數據庫可以確定100或1756400最小的那個之前，必須計算所有結果。 EXPLAIN QUERY PLAN輸出：

 
0,0,0,SCAN TABLE masterInfo AS m 
0,1,1,SEARCH TABLE primDescT USING INTEGER PRIMARY KEY (rowid=?) 
0,2,2,SEARCH TABLE secDescT USING INTEGER PRIMARY KEY (rowid=?) 
0,0,0,USE TEMP B-TREE FOR ORDER BY 

要刪除顯式排序步驟，你必須索引列：

CREATE INDEX pd ON primDescT(primDesc); 

而且你必須迫使數據庫使用這個索引（默認情況下，SQLite的估計查詢時忽略LIMIT成本，如果你想所有的結果，而不是使用pd指數會更快）：

SELECT ... 
FROM masterInfo m 
INNER JOIN primDescT INDEXED BY pd ON primDescT.id = m.primDescId 
--     ^^^^^^^^^^^^^ 
INNER JOIN secDescT ON secDescT.id = m.secDescId 
WHERE ... 
ORDER BY pDescr ASC 
LIMIT 100 OFFSET ...; 

 
0,0,1,SCAN TABLE primDescT USING COVERING INDEX pd 
0,1,0,SEARCH TABLE masterInfo AS m USING COVERING INDEX sqlite_autoindex_masterInfo_1 (primDescId=?) 
0,2,2,SEARCH TABLE secDescT USING INTEGER PRIMARY KEY (rowid=?) 

較大的OFFSET值總是很慢;數據庫必須計算並丟棄所有這些行。

如果您正在使用分頁，您可以用排序列上的查找替換OFFSET;這要求您保存上一頁的最後一個值：

SELECT ... 
FROM masterInfo m 
INNER JOIN primDescT INDEXED BY pd ON primDescT.id = m.primDescId 
INNER JOIN secDescT ON secDescT.id = m.secDescId 
WHERE primDesc > :LastValue 
-- ^^^^^^^^^^^^^^^^^^^^^ 
    AND ... 
ORDER BY pDescr ASC 
LIMIT 100 /* no offset */;