BQ命令行：查詢太大

Question

我正在從命令行運行查詢以存儲在新表中。在這個查詢中，我有幾個子查詢，每個子查詢使用TABLE_DATE_RANGE訪問多個表。

對於每個表存根，每天有一個表。所以有4個子查詢，每個accessign 180個表（在兩個TABLE_DATE_RANGE查詢中爲90天）。這總計達720個表格。所以我不應該超出1k表的限制。

我已經超過了1K表格限制，並得到一個錯誤，指出「太多表」或類似。

但是這個查詢給我錯誤「查詢太大」。正如你可以看到下面，我確實允許大的結果。有沒有人知道這個解決方案？

bq query -n0 --allow_large_results --replace --destination_table="cdate-prod:crm_adhoc.tmp_email_details_event_date" 'select event_date 
,contact_id 
,message_name 
,message_name_join 
,message_id 
,email 
,REGEXP_EXTRACT(email,r'([^@]*$)') as email_domain 
,REGEXP_EXTRACT(REGEXP_EXTRACT(email,r'([^@]*$)'),r'(^[^\.]*)') as email_provider 
,sent 
,sent_unique_hlp 
,open 
,open_unique_hlp 
,open_unique_msg_hlp 
,click 
,click_unique_hlp 
,click_unique_msg_hlp 
,soft_bounce 
,medium_bounce 
,hard_bounce 
,activity 
,type 
,case when type = 1 then 'PPM' 
     when type = 2 then 'NPM' 
     when type = 3 then 'PENDING' 
     when type = 4 then 'CB' 
     when type = 5 then 'REDEBIT' 
     when type = 6 then 'INTCO' 
     when type = 7 then 'EXTCO' 
     else 'XX' 
end as type_str 
from 

(select send_date as event_date 
,contact_id 
,message_name 
,substr(message_name,7) as message_name_join 
,message_id 
,email 
, 1 as sent 
, contact_id as sent_unique_hlp 
, 0 as open 
, string('') as open_unique_hlp 
, string('') as open_unique_msg_hlp 
, 0 as click 
, string('') as click_unique_hlp 
, string('') as click_unique_msg_hlp 
, 0 as soft_bounce 
, 0 as medium_bounce 
, 0 as hard_bounce 
, IFNULL(activity,0) as activity 
, IFNULL(type,0) as type 
from TABLE_DATE_RANGE(crm_data.campaign_messages,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day")), 
    TABLE_DATE_RANGE(crm_data.interface_messages,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day"))) ms, 

(select open_date as event_date 
,contact_id 
,message_name 
,substr(message_name,7) as message_name_join 
,message_id 
,email 
, 0 as sent 
, string('')as sent_unique_hlp 
, 1 as open 
, contact_id as open_unique_hlp 
, concat(contact_id,string(TIMESTAMP_TO_MSEC(send_date))) open_unique_msg_hlp 
, 0 as click 
, string('') as click_unique_hlp 
, string('') as click_unique_msg_hlp 
, 0 as soft_bounce 
, 0 as medium_bounce 
, 0 as hard_bounce 
, IFNULL(activity,0) as activity 
, IFNULL(type,0) as type 
from TABLE_DATE_RANGE(crm_data.interface_openings,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day")), 
    TABLE_DATE_RANGE(crm_data.campaign_openings,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day"))) op, 

(select click_date as event_date 
,contact_id 
,message_name 
,substr(message_name,7) as message_name_join 
,message_id 
,email 
, 0 as sent 
, string('')as sent_unique_hlp 
, 0 as open 
, string('') as open_unique_hlp 
, string('') as open_unique_msg_hlp 
, 1 as click 
, contact_id as click_unique_hlp 
, concat(contact_id,string(TIMESTAMP_TO_MSEC(send_date))) click_unique_msg_hlp 
, 0 as soft_bounce 
, 0 as medium_bounce 
, 0 as hard_bounce 
, IFNULL(activity,0) as activity 
, IFNULL(type,0) as type 
from TABLE_DATE_RANGE(crm_data.interface_clicks,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day")), 
    TABLE_DATE_RANGE(crm_data.campaign_clicks,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day"))) cl, 

(select bounce_date as event_date 
,contact_id 
,message_name 
,substr(message_name,7) as message_name_join 
,message_id 
,email 
, 0 as sent 
, string('')as sent_unique_hlp 
, 0 as open 
, string('') as open_unique_hlp 
, string('') as open_unique_msg_hlp 
, 0 as click 
, string('') as click_unique_hlp 
, string('') as click_unique_msg_hlp 
,case when bounce_category = 1 then 1 end soft_bounce 
,case when bounce_category = 2 then 1 end medium_bounce 
,case when bounce_category in (3,4,5) then 1 end hard_bounce 
, IFNULL(activity,0) as activity 
, IFNULL(type,0) as type 
from TABLE_DATE_RANGE(crm_data.interface_bounces,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day")), 
    TABLE_DATE_RANGE(crm_data.campaign_bounces,date_add(CURRENT_DATE(),-90,"day"),date_add(CURRENT_DATE(),-1,"day"))) bo' 

Waiting on bqjob_r71fbdcc95fa950e5_0000014f82f52d18_1 ... (0s) Current status: RUNNING 
Waiting on bqjob_r71fbdcc95fa950e5_0000014f82f52d18_1 ... (1s) Current status: RUNNING 
Waiting on bqjob_r71fbdcc95fa950e5_0000014f82f52d18_1 ... (1s) Current status: DONE 
Error in query string: Error processing job 
'cdate-prod:bqjob_r71fbdcc95fa950e5_0000014f82f52d18_1': Query too large 

Answer 1

我的猜測，據我所知：

• 查詢只能達到X字符。此處顯示的查詢短於此，但...

• TABLE_DATE_RANGE通過內部擴展查詢以顯式包含所有範圍內的表名稱。這通常很好，但...

• 此查詢引用720個表。該查詢將通過明確提及720 *長度（table_name）進行擴展。這超過了極限。

建議：你可以將舊錶格合併到每月實體而不是每日實體嗎？