需要使用SSIS

Question

我有一個巨大的數據集在SQL Server（約32萬行），這是考慮到實現和源代碼如下返回巨大的數據集（3200萬）加載到表：

SELECT Idenitifier = ISNULL(mle.MIdeer, mle.Ider) + em.MemberId, 
     EffectiveDate = ISNULL(em.EffectiveDate, 
           (SELECT TOP 1 EffectiveDate 
           FROM c 
           WHERE SourceType = em.SourceType 
            AND GroupNumber = em.GroupNumber 
            AND ISNULL(GroupDivision, '') = 
             ISNULL(em.GroupDivision, ''))) 
FROM a em 
     JOIN b mle 
     ON mle.Identifier = em.GroupNumber + ISNULL('-' + em.GroupDivision, '') 

      -- Filter invalid legal entities 
      AND ISNULL(mle.Filter, 0) = 0 

     --- Gets a resultset of 531798 rows 

     CROSS JOIN -- this returns 63 rows , so 
     -- I am presuming 531798*63 rows here. 
     (SELECT * 
     FROM map 
     WHERE domaintype = 'MC')b; 

我需要使用SSIS將此數據集加載到表中。在1600萬行之後，當我給出select * from <<view>>時，我得到了sql server中的內存異常的system.out。如何使用SSIS在表格中加載此數據集，避免此異常。

還有哪些其他更好的方法可以有效地執行此查詢，因爲它需要超過30分鐘才能運行？

Answer 1

我還是通過這個思維，但是你可能需要分開CROSS JOIN：

;WITH cte AS (SELECT ISNULL(mle.MIdeer, mle.Ider) + em.MemberId AS Idenitifier 
         , ISNULL(em.EffectiveDate, 
           (SELECT TOP 1 
             EffectiveDate 
           FROM c 
           WHERE SourceType = em.SourceType 
             AND GroupNumber = em.GroupNumber 
             AND ISNULL(GroupDivision, '') = ISNULL(em.GroupDivision, 
                       '') 
           )) AS EffectiveDate 
       FROM a em 
       JOIN b mle ON mle.Identifier = em.GroupNumber + ISNULL('-'+ em.GroupDivision,'') 
          AND ISNULL(mle.Filter, 0) = 0) 
SELECT * 
FROM cte 
CROSS JOIN (SELECT * 
      FROM map 
      WHERE domaintype = 'MC') b;