問題與SSIS包

在經過幾個月下面描述的過程沒有任何問題已經工作加載數據時，瞭解間歇不一致絕大多數已運行時間（2008 R2）。然而，我們有三個錯誤連接的數據。問題是，是什麼造成了這種情況，我該如何補救？

DATA_PreImp 

    sourceid col01 col02 col03 col04 col... 
    100001  John Smith 
    100002  Calvin Klein 
    100003  Peter Parker 
    100004  Moe  Greene

一般所呈現的結果是該屬性被連接到Items_Main正確，但有時（小於1％）的順序進行加擾，使得COL01的值不被連接到相同的Items_Main爲的值其餘的列。

任何有關這是什麼導致的見解將不勝感激。

的數據移動過程

我們有一個SSIS包從稱爲DATA_PreImp到（基於屬性）由三個相關的表的結構的平坦表傳輸數據。

Items_Main應該包含一個行用於DATA_PreImp每一行
Items_Featurevalues包含用於DATA_PreImp一個行的每個列值一行
Items_MainRel包含Items_Main和Items_FeatureValues

的之間的連接SSIS包中的第一步是將DATA_PreImp中的數據插入到Items_Main中，並將生成的標識符插入到空DATA_PreImpMappingTMP表中的TARGET_ID列中。

insert into items_main(creationdate, status) 
output inserted.itemid into DATA_PreImpMappingTMP(TARGET_ID) 
select getdate(), '0' from data_preimp 
order by sourceid asc;

SSIS包第二步填充TARGET_ID（ITEMID最初）的Items_MainRel表和用於特徵的標識符（在這種情況下爲5）。

insert into items_mainrel(itemid, featureid) 
output inserted.itemrelid into DATA_PreImpMapping2TMP(INDREL_ID) 
select TARGET_ID, 5 from DATA_PreImpMappingTMP 
order by TARGET_ID asc;

的第三步是從與所述DATA_PreImp SOURCE_ID填寫DATA_PreImpMapping2TMP表中的列SOURCE_ID。

with cte as (select sourceid, row_number() over (order by sourceid asc) as row from data_preimp) 
update m set m.SOURCE_ID = s.sourceid, m.FEAT_ID = 5 
from DATA_PreImpMapping2TMP as m 
join cte as s on s.row = m.ROW;

的最後一步是填充Items_FeatureValues表與來自DATA_PreImpMapping2TMP和DATA_PreImp數據。

insert into items_featurevalues(itemrelid, languageid, fnvarchar) 
select DATA_PreImpMapping2TMP.INDREL_ID, 0, data_preimp.col01 
from DATA_PreImpMapping2TMP 
join data_preimp on (DATA_PreImpMapping2TMP.SOURCE_ID = data_preimp.sourceid) 
where FEAT_ID = 5

數據表結構

這裏是什麼是需要創建方案：

CREATE TABLE [dbo].[DATA_PreImp](
    [sourceid] [bigint] IDENTITY(1,1) NOT NULL, 
    [col01] [nvarchar](500) NULL, 
    [col02] [nvarchar](500) NULL, 
    [col03] [nvarchar](500) NULL, 
    [col04] [nvarchar](500) NULL, 
    [col05] [nvarchar](500) NULL, 
    [col06] [nvarchar](500) NULL, 
    [col07] [nvarchar](500) NULL, 
    [col08] [nvarchar](500) NULL, 
    [col09] [nvarchar](500) NULL, 
    [col10] [nvarchar](500) NULL, 
CONSTRAINT [PK_DATA_PreImp] PRIMARY KEY CLUSTERED 
(
    [sourceid] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] 
) ON [PRIMARY] 


CREATE TABLE [dbo].[DATA_PreImpMappingTMP](
    [ROW] [int] IDENTITY(1,1) NOT NULL, 
    [TARGET_ID] [int] NULL, 
PRIMARY KEY CLUSTERED 
(
    [ROW] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] 
) ON [PRIMARY] 


CREATE TABLE [dbo].[Items_Main](
    [Itemid] [int] IDENTITY(1,1) NOT NULL, 
    [creationDate] [smalldatetime] NOT NULL, 
    [status] [int] NOT NULL, 
    [purchdate] [smalldatetime] NULL, 
    [logindate] [smalldatetime] NULL, 
CONSTRAINT [PK_Items_Main] PRIMARY KEY CLUSTERED 
(
    [Itemid] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] 
) ON [PRIMARY] 


CREATE TABLE [dbo].[DATA_PreImpMapping2TMP](
    [ROW] [int] IDENTITY(1,1) NOT NULL, 
    [SOURCE_ID] [int] NULL, 
    [INDREL_ID] [int] NULL, 
    [FEAT_ID] [int] NULL, 
PRIMARY KEY CLUSTERED 
(
    [ROW] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] 
) ON [PRIMARY] 

CREATE TABLE [dbo].[Items_Features](
    [featureId] [int] IDENTITY(1,1) NOT NULL, 
    [featureRef] [varchar](15) NOT NULL, 
    [featureName] [varchar](50) NOT NULL, 
    [creationDate] [smalldatetime] NOT NULL, 
    [status] [int] NOT NULL, 
    [fieldType] [varchar](50) NOT NULL, 
    [featureType] [int] NOT NULL, 
    [featureDesc] [varchar](500) NULL, 
CONSTRAINT [PK_Items_Features] PRIMARY KEY CLUSTERED 
(
    [featureId] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY] 


CREATE TABLE [dbo].[Items_MainRel](
    [ItemRelId] [int] IDENTITY(1,1) NOT NULL, 
    [Itemid] [int] NOT NULL, 
    [featureId] [int] NOT NULL, 
CONSTRAINT [PK_Items_MainRel] PRIMARY KEY CLUSTERED 
(
    [ItemRelId] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] 
) ON [PRIMARY] 

GO 

CREATE TABLE [dbo].[Items_FeatureValues](
    [valueId] [int] IDENTITY(1,1) NOT NULL, 
    [ItemRelId] [int] NOT NULL, 
    [languageId] [int] NOT NULL, 
    [FnVarChar] [nvarchar](250) NULL, 
    [FInt] [int] NULL, 
    [FImage] [int] NULL, 
    [FNText] [ntext] NULL, 
    [FSmallDateTime] [smalldatetime] NULL, 
CONSTRAINT [PK_Items_FeatureValues] PRIMARY KEY CLUSTERED 
(
    [valueId] ASC 
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY] 
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] 

GO  

ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Features] FOREIGN KEY([featureId]) 
REFERENCES [dbo].[Items_Features] ([featureId]) 
GO 

ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Features] 
GO 

ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Main] FOREIGN KEY([Itemid]) 
REFERENCES [dbo].[Items_Main] ([Itemid]) 
GO 

ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Main] 
GO 


ALTER TABLE [dbo].[Items_FeatureValues] WITH CHECK ADD CONSTRAINT [FK_Items_FeatureValues_Items_MainRel] FOREIGN KEY([ItemRelId]) 
REFERENCES [dbo].[Items_MainRel] ([ItemRelId]) 
ON DELETE CASCADE 
GO 

ALTER TABLE [dbo].[Items_FeatureValues] CHECK CONSTRAINT [FK_Items_FeatureValues_Items_MainRel] 
GO 


INSERT INTO DATA_PreImp (col01,col02,col03,col04) 
VALUES('John', 'Smith', '1964', 'NewYork'), 
     ('Calvin', 'Klein', '1960', 'Washington D. C.'), 
     ('Peter', 'Parker', '1974', 'Losangles'), 
     ('Moe', 'Greene', '1928', 'Lasvegas') 


INSERT INTO Items_Features (featureRef, featureName, creationDate, [status], fieldType, featureType, featureDesc) 
VALUES ('firstname','First_Name', GETDATE(), 0, 'FnVarChar', 3, 'FirstName'), 
    ('lastname','Last_Name', GETDATE(), 0, 'FnVarChar', 3, 'LastName'), 
    ('Birth','Birth', GETDATE(), 0, 'FnVarChar', 3, 'Birth'), 
    ('City','City', GETDATE(), 0, 'FnVarChar', 3, 'City')

來源

2013-03-07 ahdaniels

我在第二個腳本上獲得外鍵違規。 dbo中沒有記錄。具有featureID = 5的Items_Features（如硬編碼到該腳本中）。應該是腳本中使用的另一個featureID還是dbo.Items_Features中缺少記錄？ – Aphillippe 2013-03-27 09:03:58

你在插入中使用交易嗎？ SSIS包中的錯誤處理是否會拋出任何有用的數據？是否有其他進程在同一個db/tables上同時運行，可能會導致併發問題？ – Valkyrie 2013-04-24 14:01:59

這個複雜的東西，沒有真正的包裝和所有的內部工作，找到問題將是相當具有挑戰性的。我們需要更多的信息...... – mrkb80 2013-05-07 17:33:25

問題是計算列CUST_CD。經過大量研究，似乎BULK INSERT不喜歡複雜的計算類型（請參見Using SQL Server spatial types in SSIS data load）。解決方案是刪除計算列，並使其成爲varchar（20）NULL。然後我創建了一個新的執行SQL任務，用計算的值更新任何NULL行。

來源

2013-06-07 19:00:41

回答

相關問題