RedShift複製命令返回

Question

我們可以通過複製命令獲取插入的行數嗎？有些記錄可能會失敗，那麼成功插入的記錄數是多少？

我有一個json對象在Amazon S3中的文件，並嘗試使用複製命令將數據加載到Redshift中。我如何知道成功插入了多少條記錄以及有多少條失敗？

Answer 1

載入一些示例性數據：

db=# copy test from 's3://bucket/data' credentials '' maxerror 5; 
INFO: Load into table 'test' completed, 4 record(s) loaded successfully. 
COPY 

db=# copy test from 's3://bucket/err_data' credentials '' maxerror 5; 
INFO: Load into table 'test' completed, 1 record(s) loaded successfully. 
INFO: Load into table 'test' completed, 2 record(s) could not be loaded. Check 'stl_load_errors' system table for details. 
COPY 

然後以下查詢：

with _successful_loads as (
    select 
     stl_load_commits.query 
     , listagg(trim(filename), ', ') within group(order by trim(filename)) as filenames 
    from stl_load_commits 
    left join stl_query using(query) 
    left join stl_utilitytext using(xid) 
    where rtrim("text") = 'COMMIT' 
    group by query 
), 
_unsuccessful_loads as (
    select 
     query 
     , count(1) as errors 
    from stl_load_errors 
    group by query 
) 
select 
    query 
    , filenames 
    , sum(stl_insert.rows)   as rows_loaded 
    , max(_unsuccessful_loads.errors) as rows_not_loaded 
from stl_insert 
inner join _successful_loads using(query) 
left join _unsuccessful_loads using(query) 
group by query, filenames 
order by query, filenames 
; 

給予：

query |     filenames     | rows_loaded | rows_not_loaded 
-------+------------------------------------------------+-------------+----------------- 
45597 | s3://bucket/err_data.json      |   1 |    2 
45611 | s3://bucket/data1.json, s3://bucket/data2.json |   4 | 
(2 rows)