插入語句的性能改進

在我的ASP.NET MVC應用程序中，我正在運行一些插入，它可以插入10000行或更多行並更新其他幾行。這個過程需要很長時間，但我無法逃避插入，因爲這正是我被要求做的。現在我正在運行Sql Server Profiler，它需要將近20分鐘才能插入這一堆行。我怎麼可能改善這一行動的表現？插入語句的性能改進

（我使用LINQ到SQL來將數據插入到數據庫。）

這是方法做的代碼插入數據：

[AcceptVerbs(HttpVerbs.Post)] 
     public ActionResult SaveEvent(int id) 
     { 
      int eventID= 0; 
      var query = from q in context.InventoryGoods 
         where q.ParentId == id && q.Action.HasValue && q.ActionOn.HasValue == false 
         select q; 

      var stockType = from q in context.Inventory 
          where q.Id == id 
          select q.StockType; 

      if (query.Count() > 0) 
      { 
       foreach (var i in query) 
       { 
         switch (i.Action.Value) 
         { 
          case (int)InventoryGoodsActionEnum.AdjustLocation: 

           Guid guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "LO", 
             Lid = i.LidObtained, 
             Comments = "Inventário "+i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 



           break; 

          case (int)InventoryGoodsActionEnum.AdjustQuantity: 

           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now, 
              EventOn = DateTime.Now, 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0) != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now, 
              EventOn = DateTime.Now, 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0)), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 


           break; 

          case (int)InventoryGoodsActionEnum.AdjustQuantityLocation: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "LO", 
             Lid = i.LidExpected, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 
           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0) != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - (i.QuantityExpected.HasValue ? i.QuantityExpected : 0)), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 



           break; 

          case (int)InventoryGoodsActionEnum.AdjustStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 


           break; 
          case (int)InventoryGoodsActionEnum.AdjustLocationStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now.AddSeconds(1), 
             EventOn = DateTime.Now.AddSeconds(1), 
             Type = "LO", 
             Lid = i.LidExpected, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 



            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 



           break; 
          case (int)InventoryGoodsActionEnum.AdjustQuantityStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 

           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - i.QuantityExpected != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - i.QuantityExpected), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 


           break; 
          case (int)InventoryGoodsActionEnum.AdjustQuantityLocationStockType: 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now, 
             EventOn = DateTime.Now, 
             Type = "ST", 
             StockType = stockType.First().Value, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 

            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 

           if (!i.QuantityObtained.HasValue) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityExpected * -1), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 

           } 
           else if ((i.QuantityObtained - i.QuantityExpected != 0)) 
           { 
            guid = Guid.NewGuid(); 

            using (var scope = new TransactionScope()) 
            { 
             GoodsEvent ge = new GoodsEvent() 
             { 
              Gid = i.Gid, 
              Guid = guid, 
              CreatedOn = DateTime.Now.AddSeconds(1), 
              EventOn = DateTime.Now.AddSeconds(1), 
              Type = "AQ", 
              Quantity = (short)(i.QuantityObtained.Value - i.QuantityExpected), 
              Comments = "Inventário " + i.ParentId, 
              UserId = GetUserId(), 
             }; 

             context.GoodsEvent.InsertOnSubmit(ge); 

             context.SubmitChanges(); 
             eventID = ge.Id; 
             Repository.SetActionOn(i.Id, eventID); 

             scope.Complete(); 
            } 
           } 

           guid = Guid.NewGuid(); 

           using (var scope = new TransactionScope()) 
           { 
            GoodsEvent ge = new GoodsEvent() 
            { 
             Gid = i.Gid, 
             Guid = guid, 
             CreatedOn = DateTime.Now.AddSeconds(2), 
             EventOn = DateTime.Now.AddSeconds(2), 
             Type = "LO", 
             Lid = i.LidExpected, 
             Comments = "Inventário " + i.ParentId, 
             UserId = GetUserId(), 
            }; 



            context.GoodsEvent.InsertOnSubmit(ge); 

            context.SubmitChanges(); 
            eventID = ge.Id; 
            Repository.SetActionOn(i.Id, eventID); 

            scope.Complete(); 
           } 




           break; 
        } 
       } 
      } 
      else 
      { 
       var lista = from q in context.InventoryGoods 
          where q.ParentId == id 
          select q; 

       Repository.EvaluateActions(lista.ToList()); 

       SaveEvent(id); 
      } 


      using (var scope = new TransactionScope()) 
      { 
       var thisInventory = from i in context.Inventory 
            where i.Id == id 
            select i; 

       thisInventory.First().State = (int)InventoryStateEnum.Verified; 

       context.SubmitChanges(); 

       scope.Complete(); 
      } 

      Status.Info(string.Format("Acções aplicadas com sucesso.")); 
      return RedirectToAction("Details", new { id }); 
     } 



public void SetActionOn(int id, int eventID) 
     { 
      var InventoryGoods = from i in context.InventoryGoods 
           where i.Id == id 
           select i; 



      using (var scope = new TransactionScope()) 
      { 
       InventoryGoods.First().ActionOn = DateTime.Now; 

       InventoryGoodsEvents ige = new InventoryGoodsEvents 
       { 
        EventId = eventID, 
        InventoryGood = InventoryGoods.First().Id, 
       }; 

       context.InventoryGoodsEvents.InsertOnSubmit(ige); 

       scope.Complete(); 
      } 
     }

來源

2010-07-21 Hallaghan

這種方法是怎麼回事：Repository.SetActionOn（i.Id，eventID）; – 2010-07-21 11:42:07

我會在我的主要帖子上張貼它。 – Hallaghan 2010-07-21 11:45:34

停止使用var這麼多。

這會運行查詢兩次（在sqlprofiler中觀察）。

if (query.Count() > 0) 
{ 
    foreach (var i in query)

使用此選項可避免多次運行查詢。

List<InventoryGoods> rows = query.ToList();

你的代碼有很多重複。另外，您在每個提交的上下文中嘗試做的太少。您無故控制交易範圍。

foreach(InventoryGood i in rows) 
{ 
    InventoryGoodsEvent ige = new InventoryGoodsEvent() 
    //this will attach ige to the object graph tracked by context 
    // which is sufficient to insert ige when submitchanges is called. 
    ige.InventoryGood = i; 

    GoodsEvent ge = GetGoodsEvent(i); //all that conditional logic in there. 
    //this will attach ge to the object graph tracked by context 
    // which will both insert ge and update ige with ge's id when submitchanges is called. 
    ige.GoodsEvent = ge; 

    i.ActionOn = DateTime.Now; 
    //to submit each row, uncomment this. 
    //context.SubmitChanges(); 
} 
//to submit all rows at once, use this. 
context.SubmitChanges();

如果InventoryGoodEgvents沒有這些關係屬性，請進入設計器並添加關聯以創建它們。

一旦你有了這樣的代碼，那麼你就可以決定在一個事務中做什麼樣的改變。我喜歡爲每筆交易插入~100條記錄。如果每個事務使用1條記錄，則創建每個事務的開銷很大。如果每個事務使用1,000,000行，則長時間運行事務的開銷很大。

這東西很難學，但要堅持下去。

還有一件事：批量插入不適用於多對多表。

來源

2010-07-21 12:11:52

<主觀評論警報>這裏使用var的方式沒有任何問題。它與Linq查詢結合在一起，每個人都知道返回IEnumerable或者你選擇的任何東西，並且他還與新關鍵字結合使用 - 這比冗長的TransactionScope scope = new TransactionScope（）少得多。 Var可以被過度使用，但在這裏肯定是適當的。 – mattmc3 2010-07-21 17:02:11

有些特定的聲明需要var。使用var之外的是風格問題（如你所說，主觀）。您選擇的案例僅保存13個字符。也許如果線路沒有針對長度進行優化，那麼就不會使用這麼多次，最終可以節省很多更多的字符。在理解查詢語法結果分配的情況下，如你所說的「或其他」。「或者不管」是問題。 – 2010-07-22 02:18:16

什麼DAL你使用EF，L2S，ADO.net還是其他？插入不應該花很多時間來完成。您可以將它們插入本地緩存，稍後再提交更改。

來源

2010-07-21 11:01:04 Wyvern

我們使用的是ADO.NET。 – Hallaghan 2010-07-21 11:05:31

嘗試插入8500行後，系統花30分鐘完成操作。 – Hallaghan 2010-07-21 11:16:52

好吧，明白了。請看下面的這個模式： context.GoodsEvent.InsertOnSubmit（ge）; context.SubmitChanges（）; 只需移動「context.SubmitChanges（）;」甚至超出了foreach循環的範圍。 SubmitChanges是一個耗時的操作，它同步本地緩存和遠程Sql DB數據狀態。如果你插入8500行，它會同步8500次？但將SubmitChanges（）移出循環後，它只同步一次。 – Wyvern 2010-07-21 11:36:05

Linq-to-sql實際上並沒有設計用於在同一批次中將多條記錄插入數據庫。它會做insert聲明insert聲明這真的很慢。我建議你在任何你知道需要支持這個插入的地方使用SqlBulkCopy對象來代替你的Linq-to-sql類。如果您需要它們進行對象驗證，您甚至可以使用相同的L2S類，但只需將它們轉儲到1000行塊中的DataTable中，並讓SqlBulkCopy執行您的實際插入。你甚至可以谷歌L2S和SqlBulkCopy，看看那裏有什麼擴展方法或其他集成。你不是第一個遇到這個問題的人。

來源

2010-07-21 11:24:40 mattmc3

+1'SqlBulkCopy' – abatishchev 2010-07-21 11:27:16

你看過我上面的方法的代碼了嗎？你認爲我可以改進它以使其更快嗎？我想知道我是否可以運行這些「context.SubmitChanges（）;」只是在我的開關結束時，或者如果我現在需要運行他們。你怎麼看？ – Hallaghan 2010-07-21 11:30:00

當我發佈我的答案時，它並不存在，但乍一看有很多重複的代碼。在優化插入性能之前，這對於重構**可能已經成熟。 – mattmc3 2010-07-21 12:10:20

插入語句的性能改進

回答

相關問題