Postgres - 大jsonb列的性能

Question

我們在我們的數據庫表中使用了Postgres jsonb類型。表結構如下所示：

CREATE TABLE T (
    id UUID NOT NULL PRIMARY KEY, 
    payload JSONB 
); 

CREATE INDEX ON T USING gin (payload jsonb_path_ops); 

Payload是一個複雜的JSON字符串。下面是一個例子：

{ 
    "business": { 
     "taxId": "626642071", 
     "legalName": "Jon's Deli", 
     "phoneNumbers": [ 
      { 
       "phoneType": "Business", 
       "telephoneNumber": "8384407555" 
      }, 
      { 
       "phoneType": "Work", 
       "telephoneNumber": "6032255248" 
      } 
     ], 
     "addresses": [ 
      { 
       "city": "San Francisco", 
       "state": "CA", 
       "postalCode": "94101", 
       "countryCode": "USA", 
       "addressLine1": "123 Market St" 
      } 
     ] 
    }, 
    "stakeholders": [ 
     { 
      "person": { 
       "taxId": "540646815", 
       "firstName": "GdXFouh", 
       "lastName": "IlUAcgCGz", 
       "dateOfBirth": "1980-12-11", 
       "emailAddress": "jywxsijgix@qaqmlz.com", 
       "phoneNumbers": [ 
        { 
         "phoneType": "Mobile", 
         "telephoneNumber": "4901371573" 
        } 
       ], 
       "addresses": [ 
        { 
         "city": "San Francisco", 
         "state": "CA", 
         "postalCode": "94101", 
         "countryCode": "USA", 
         "addressLine1": "123 Market St" 
        } 
       ] 
      } 
     } 
    ] 
} 

注意phoneNumbers，addresses和stakeholders是數組，這意味着可以有陣列中的多個元素。

我嘗試在表中插入一百萬行。生成​​的每個字段隨機。最初測試程序運行速度非常快。但插入大約800,000行後，每隔1000行就卡住 - 插入1000行，然後測試程序掛起2分鐘，然後返回並插入另外1000行，...

我們懷疑這個是由大量的jsonb索引更新引起的。因爲單行索引中有很多字段需要更新。我們只是想確認是否有人遇到同樣的問題。

---

其實我們並不需要索引整個​​列。只需要某些字段：business->taxId,business->phoneNumbers-> telephoneNumber,stakeholders->person->taxId和stakeholders->person->emailAddress。

我已經嘗試了以下兩個指標：

CREATE INDEX ON T USING gin ((payload->'business'->'taxId') jsonb_path_ops); 
CREATE INDEX ON T USING gin ((payload ->'stakeholders'->'person'->'taxId') jsonb_path_ops); 

並運行兩個語句：

explain select * from T where payload->'business'->'taxId' @> '"123456789"'; (1) 
explain select * from T where payload->'stakeholders'->'person'->'taxId' @> '"123456789"'; (2) 

第一個語句使用索引。但第二個是做一個非常緩慢的全表掃描。這就是爲什麼我們轉向索引整個​​列。

---

任何建議是值得歡迎的。

順便說一句，我們正在使用Postgres 9.5.4。

Answer 1

您的疑問：

select * from T where payload->'stakeholders'->'person'->'taxId' @> '"123456789"'; 

不起作用。這是因爲'利益相關者'是數組。工作查詢是：

select * from T where payload->'stakeholders' @> '[{"person": {"taxId": "54"}}]'::jsonb 

但在這種情況下，postgres可以使用整個利益相關者的使用指數。

             QUERY PLAN              
------------------------------------------------------------------------------------------------------------------------ 
Bitmap Heap Scan on t (cost=1388.08..1425.90 rows=10 width=36) (actual time=1.959..1.959 rows=1 loops=1) 
    Recheck Cond: ((payload -> 'stakeholders'::text) @> '[{"person": {"taxId": "54"}}]'::jsonb) 
    Heap Blocks: exact=1 
    -> Bitmap Index Scan on t_expr_idx3 (cost=0.00..1388.08 rows=10 width=0) (actual time=1.946..1.946 rows=1 loops=1) 
     Index Cond: ((payload -> 'stakeholders'::text) @> '[{"person": {"taxId": "54"}}]'::jsonb) 
Planning time: 0.071 ms 
Execution time: 1.978 ms 

對於使用更具體的指標我用修改的辦法：How do you create a Postgresql JSONB array in array index?

CREATE OR REPLACE FUNCTION extract_taxids(a_json jsonb). 
RETURNS jsonb AS $BODY$ 
    SELECT jsonb_agg(j) FROM (SELECT jsonb_array_elements(a_json->'stakeholders')->'person'->'taxId' AS j) AS j 
$BODY$ LANGUAGE sql IMMUTABLE; 
CREATE INDEX ON T USING gin (extract_taxids(payload)); 

而且vuala：

EXPLAIN ANALYZE select * from T where extract_taxids(payload) @> '["54"]'; 

                  QUERY PLAN               
-------------------------------------------------------------------------------------------------------------------------------- 
Bitmap Heap Scan on t (cost=12.08..52.38 rows=10 width=36) (actual time=0.101..0.102 rows=1 loops=1) 
    Recheck Cond: (extract_taxids(payload) @> '["54"]'::jsonb) 
    Heap Blocks: exact=1 
    -> Bitmap Index Scan on t_extract_taxids_idx (cost=0.00..12.07 rows=10 width=0) (actual time=0.008..0.008 rows=1 loops=1) 
     Index Cond: (extract_taxids(payload) @> '["54"]'::jsonb) 
Planning time: 0.128 ms 
Execution time: 0.117 ms