2017-10-10 114 views
1

我試圖在SQL Server 2008中優化以下查詢。執行計劃顯示此查詢相對於批處理爲90%。插入顯示75%。我正在尋找一個選項來進行批量插入,併發插入等。有人可以拋開光速加快這一進程。我可以做SqlBulkCopy或使用OpenRowset功能嗎?我已確保有所有在那裏的條件提高SQL Server中插入的速度

INSERT INTO [xxxxxxx].[dbo].[xxxxxxx] WITH (Tablock) 
    SELECT 
     s.companyId, 
     ti.tickerSymbol, 
     s.securityName, 
     ex.exchangeName, 
     cur.currencyName, 
     primaryFlag = CASE 
         WHEN ti.primaryFlag = 1 AND s.primaryFlag = 1 
          THEN 1 
         ELSE 0 
         END, 
     ti.tradingItemId, 
     peq.pricingDate, peq.priceOpen, 
     peq.priceHigh, peq.priceLow, peq.priceMid, 
     peq.priceClose, peq.priceBid, peq.priceAsk, 
     peq.volume, peq.adjustmentFactor, peq.VWAP, 
     mc.marketCap, mc.TEV, mc.sharesOutstanding 
    FROM 
     ciqsecurity s 
    INNER JOIN 
     CoreReferenceStaging.dbo.MarketDataTemp1 a ON a.companyId = s.companyId 
    INNER JOIN 
     ciqtradingitem ti ON s.securityid = ti.securityid 
    LEFT JOIN 
     ciqpriceequity peq ON peq.tradingitemid = ti.tradingitemid 
    LEFT JOIN 
     CoreReferenceStaging.dbo.MarketDataTemp2 mc ON (mc.companyId= s.companyId AND mc.pricingDate = peq.pricingDate) 
    INNER JOIN 
     ciqExchange ex ON ex.exchangeId = ti. exchangeId 
    INNER JOIN 
     ciqCurrency cur ON cur.currencyid = ti.currencyId 
    ORDER BY 
     peq.pricingDate DESC 

我試圖切片INSERT INTO批次字段的索引,但它似乎需要更長的時間。對於e.g以下

DECLARE @BatchSize int = 1000 

WHILE 1 = 1 
BEGIN 

INSERT INTO [xxxxxx].[dbo].[xxxxxxx] with (Tablock) 
select 
    s.companyId, 
    ti.tickerSymbol, 
    s.securityName, 
    ex.exchangeName, 
    cur.currencyName, 
    primaryFlag = case when ti.primaryFlag = 1 and s.primaryFlag = 1 then 1 else 0 end, 
    ti.tradingItemId, 
    peq.pricingDate, 
    peq.priceOpen, 
    peq.priceHigh, 
    peq.priceLow, 
    peq.priceMid, 
    peq.priceClose, 
    peq.priceBid, 
    peq.priceAsk, 
    peq.volume, 
    peq.adjustmentFactor, 
    peq.VWAP, 
    mc.marketCap, 
    mc.TEV, 
    mc.sharesOutstanding 
from ciqsecurity s 
    inner join CoreReferenceStaging.dbo.MarketDataTemp1 a on a.companyId = s.companyId 
    inner join ciqtradingitem ti on s.securityid = ti.securityid 
    left join ciqpriceequity peq on peq.tradingitemid = ti.tradingitemid 
    left join CoreReferenceStaging.dbo.MarketDataTemp2 mc on (mc.companyId= s.companyId and mc.pricingDate = peq.pricingDate) 
    inner join ciqExchange ex on ex.exchangeId = ti. exchangeId 
    inner join ciqCurrency cur on cur.currencyid = ti.currencyId 
order by peq.pricingDate desc 

IF @@ROWCOUNT < @BatchSize BREAK END 

結果與組統計上

Table 'ciqExchange'. Scan count 17, logical reads 67, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'ciqCurrency'. Scan count 1, logical reads 4, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'MarketDataTemp1'. Scan count 10, logical reads 4, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'ciqSecurity'. Scan count 500, logical reads 53703, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'ciqTradingItem'. Scan count 17, logical reads 33082, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'Worktable'. Scan count 11382, logical reads 1193682, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'ciqPriceEquity'. Scan count 15681, logical reads 96425, physical reads 0, read-ahead reads 3, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'MarketDataTemp2'. Scan count 17, logical reads 1648, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 
Table 'Worktable'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0. 

執行計劃1

enter image description here

執行計劃2

enter image description here

+3

索引會減慢插入。你需要你創建的所有索引嗎? – Leonidas199x

+0

所以請等待...... INSERT部分緩慢,還是選擇?加插插入具有挑戰性。加強選擇以獲得所有記錄插入內存......是瓶頸。並且是select上的索引而不是目標表上的索引?禁用日誌記錄/恢復......這是事務性系統的所有開銷,可以使插入緩慢。 – xQbert

+2

是按需插入的順序? – GuidoG

回答

0

該查詢似乎很好地調整,似乎它沒有臨界點返工。它只是提取超過40-50萬條記錄,只需要11秒就可以執行。在我看來,你不能提高更多的查詢... 你可以對索引碎片進行一些檢查,如果重建它們,但你不會在5秒以內...我認爲查詢它確定,並有不是瓶頸...