1
我試圖在SQL Server 2008中優化以下查詢。執行計劃顯示此查詢相對於批處理爲90%。插入顯示75%。我正在尋找一個選項來進行批量插入,併發插入等。有人可以拋開光速加快這一進程。我可以做SqlBulkCopy
或使用OpenRowset
功能嗎?我已確保有所有在那裏的條件提高SQL Server中插入的速度
INSERT INTO [xxxxxxx].[dbo].[xxxxxxx] WITH (Tablock)
SELECT
s.companyId,
ti.tickerSymbol,
s.securityName,
ex.exchangeName,
cur.currencyName,
primaryFlag = CASE
WHEN ti.primaryFlag = 1 AND s.primaryFlag = 1
THEN 1
ELSE 0
END,
ti.tradingItemId,
peq.pricingDate, peq.priceOpen,
peq.priceHigh, peq.priceLow, peq.priceMid,
peq.priceClose, peq.priceBid, peq.priceAsk,
peq.volume, peq.adjustmentFactor, peq.VWAP,
mc.marketCap, mc.TEV, mc.sharesOutstanding
FROM
ciqsecurity s
INNER JOIN
CoreReferenceStaging.dbo.MarketDataTemp1 a ON a.companyId = s.companyId
INNER JOIN
ciqtradingitem ti ON s.securityid = ti.securityid
LEFT JOIN
ciqpriceequity peq ON peq.tradingitemid = ti.tradingitemid
LEFT JOIN
CoreReferenceStaging.dbo.MarketDataTemp2 mc ON (mc.companyId= s.companyId AND mc.pricingDate = peq.pricingDate)
INNER JOIN
ciqExchange ex ON ex.exchangeId = ti. exchangeId
INNER JOIN
ciqCurrency cur ON cur.currencyid = ti.currencyId
ORDER BY
peq.pricingDate DESC
我試圖切片INSERT INTO批次字段的索引,但它似乎需要更長的時間。對於e.g以下
DECLARE @BatchSize int = 1000
WHILE 1 = 1
BEGIN
INSERT INTO [xxxxxx].[dbo].[xxxxxxx] with (Tablock)
select
s.companyId,
ti.tickerSymbol,
s.securityName,
ex.exchangeName,
cur.currencyName,
primaryFlag = case when ti.primaryFlag = 1 and s.primaryFlag = 1 then 1 else 0 end,
ti.tradingItemId,
peq.pricingDate,
peq.priceOpen,
peq.priceHigh,
peq.priceLow,
peq.priceMid,
peq.priceClose,
peq.priceBid,
peq.priceAsk,
peq.volume,
peq.adjustmentFactor,
peq.VWAP,
mc.marketCap,
mc.TEV,
mc.sharesOutstanding
from ciqsecurity s
inner join CoreReferenceStaging.dbo.MarketDataTemp1 a on a.companyId = s.companyId
inner join ciqtradingitem ti on s.securityid = ti.securityid
left join ciqpriceequity peq on peq.tradingitemid = ti.tradingitemid
left join CoreReferenceStaging.dbo.MarketDataTemp2 mc on (mc.companyId= s.companyId and mc.pricingDate = peq.pricingDate)
inner join ciqExchange ex on ex.exchangeId = ti. exchangeId
inner join ciqCurrency cur on cur.currencyid = ti.currencyId
order by peq.pricingDate desc
IF @@ROWCOUNT < @BatchSize BREAK END
結果與組統計上
Table 'ciqExchange'. Scan count 17, logical reads 67, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqCurrency'. Scan count 1, logical reads 4, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'MarketDataTemp1'. Scan count 10, logical reads 4, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqSecurity'. Scan count 500, logical reads 53703, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqTradingItem'. Scan count 17, logical reads 33082, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Worktable'. Scan count 11382, logical reads 1193682, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'ciqPriceEquity'. Scan count 15681, logical reads 96425, physical reads 0, read-ahead reads 3, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'MarketDataTemp2'. Scan count 17, logical reads 1648, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Worktable'. Scan count 0, logical reads 0, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
執行計劃1
執行計劃2
索引會減慢插入。你需要你創建的所有索引嗎? – Leonidas199x
所以請等待...... INSERT部分緩慢,還是選擇?加插插入具有挑戰性。加強選擇以獲得所有記錄插入內存......是瓶頸。並且是select上的索引而不是目標表上的索引?禁用日誌記錄/恢復......這是事務性系統的所有開銷,可以使插入緩慢。 – xQbert
是按需插入的順序? – GuidoG