2012-12-13 28 views
2

如何在C#中將通用列表批量插入到SQL Server中,而不是循環遍歷列表並逐個插入單個項目?將泛型列表C#插入到SQL Server中

我目前有這個;

private void AddSnapshotData() 
{ 
    var password =  Cryptography.DecryptString("vhx7Hv7hYD2bF9N4XhN5pkQm8MRfxi+kogALYqwqSuo="); 
    var figDb = "ZEUS"; 
    var connString = 
    String.Format(
     "Data Source=1xx.x.xx.xxx;Initial Catalog={0};;User ID=appuser;Password={1};MultipleActiveResultSets=True", 
     figDb, password); 
    var myConnection = new SqlConnection(connString); 
    myConnection.Open(); 

    foreach (var holding in _dHoldList) 
    { 
    lbStatus.Text = "Adding information to SQL for client: " + holding.ClientNo; 
    _reports.AddZeusData("tblAllBrooksHoldingsSnapshot", "CliNo, SEDOL, ISIN, QtyHeld, DateOfSnapshot", 
         "'" + holding.ClientNo + "','" + holding.Sedol + "','" + holding.ISIN + "','" + 
         holding.QuantityHeld + "','" + DateTime.Today.ToString("yyyyMMdd") + "'", false, myConnection); 
    } 
    myConnection.Close(); 
    lbStatus.Visible = false; 
} 

dHoldListDHOLDS列表;

public class DHOLDS : ExcelReport 
{ 
public String ClientNo { get; set; } 
public String Sedol { get; set; } 
public Double QuantityHeld { get; set; } 
public Double ValueOfStock { get; set; } 
public String Depot { get; set; } 
public String ValC4 { get; set; } 
public String StockR1 { get; set; } 
public String StockR2 { get; set; } 
public Double BookCost { get; set; } 
public String ISIN { get; set; } 
} 
+0

更多信息?還是必須在「_reports」中使用一些現有的答案? – SwissCoder

+0

我可以使用我自己的方法,並且數據庫屬於我,所以我可以添加或做任何我希望的更改。 –

回答

6

可以將您的列表映射到一個數據表,然後使用SqlBulkCopy在一次插入所有行。

+1

此方法需要您將列表轉換爲DataTable,然後才能保存它,但這將是從SQL Server角度來看最快的方法... – Penfold

+0

借調。我發現這是從服務器外部將數據加載到SQL Server的最快方式。 –

+0

祝你好運,如果你想獲得身份值回來雖然。任何人都知道任何關係數據更好的方法嗎? – Nuzzolilo

0

或者,您也可以將您的列表轉換爲XML,如此博客文章所述:http://charleskong.com/blog/2009/09/insert-aspnet-objects-to-sql-server/ 但SqlBulkCopy方法似乎更好。

另一種說法:如果您想通過迭代代碼中的元素來解決它,而不是在單個事務中執行所有插入操作,則可能會提高性能。

+0

我正在做一個事務中的所有插入?所有循環都完成後我斷開連接? –

+0

我的意思是使用SQLTransaction。通過在連接上打開一個事務,如var t = connection.BeginTransaction();將其添加到執行querry並調用t.Commit()的SQLMethod中;循環後。請參閱此stackoverflow問題的信息,爲什麼它可以更快:http://stackoverflow.com/questions/5091084/ado-net-sqltransaction-improves-performance – SwissCoder

1

4年後,這是我的貢獻。我有同樣的問題,我想批量插入,但在經過某些字段是不會在數據庫中,特別是EF導航屬性,所以我寫了這個通用類:

/// <summary> 
/// This class is intended to perform a bulk insert of a list of elements into a table in a Database. 
/// This class also allows you to use the same domain classes that you were already using because you 
/// can include not mapped properties into the field excludedPropertyNames. 
/// </summary> 
/// <typeparam name="T">The class that is going to be mapped.</typeparam> 
public class BulkInsert<T> where T : class 
{ 
    #region Fields 

    private readonly LoggingService _logger = new LoggingService(typeof(BulkInsert<T>)); 
    private string _connectionString; 
    private string _tableName; 
    private IEnumerable<string> _excludedPropertyNames; 
    private int _batchSize; 
    private IEnumerable<T> _data; 
    private DataTable _dataTable; 

    #endregion 

    #region Constructor 

    /// <summary> 
    /// Initializes a new instance of the <see cref="BulkInsert{T}"/> class. 
    /// </summary> 
    /// <param name="connectionString">The connection string.</param> 
    /// <param name="tableName">Name of the table.</param> 
    /// <param name="data">The data.</param> 
    /// <param name="excludedPropertyNames">The excluded property names.</param> 
    /// <param name="batchSize">Size of the batch.</param> 
    public BulkInsert(
     string connectionString, 
     string tableName, 
     IEnumerable<T> data, 
     IEnumerable<string> excludedPropertyNames, 
     int batchSize = 1000) 
    { 
     if (string.IsNullOrEmpty(connectionString)) throw new ArgumentNullException(nameof(connectionString)); 
     if (string.IsNullOrEmpty(tableName)) throw new ArgumentNullException(nameof(tableName)); 
     if (data == null) throw new ArgumentNullException(nameof(data)); 
     if (batchSize <= 0) throw new ArgumentOutOfRangeException(nameof(batchSize)); 

     _connectionString = connectionString; 
     _tableName = tableName; 
     _batchSize = batchSize; 
     _data = data; 
     _excludedPropertyNames = excludedPropertyNames == null ? new List<string>() : excludedPropertyNames; 
     _dataTable = CreateCustomDataTable(); 
    } 

    #endregion 

    #region Public Methods 

    /// <summary> 
    /// Inserts the data with a bulk copy inside a transaction. 
    /// </summary> 
    public void Insert() 
    { 
     using (var connection = new SqlConnection(_connectionString)) 
     { 
      connection.Open(); 
      SqlTransaction transaction = connection.BeginTransaction(); 

      using (var bulkCopy = new SqlBulkCopy(connection, SqlBulkCopyOptions.Default | SqlBulkCopyOptions.KeepIdentity, transaction)) 
      { 
       bulkCopy.BatchSize = _batchSize; 
       bulkCopy.DestinationTableName = _tableName; 

       // Let's fix tons of mapping issues by 
       // Setting the column mapping in SqlBulkCopy instance: 
       foreach (DataColumn dataColumn in _dataTable.Columns) 
       { 
        bulkCopy.ColumnMappings.Add(dataColumn.ColumnName, dataColumn.ColumnName); 
       } 

       try 
       { 
        bulkCopy.WriteToServer(_dataTable); 
       } 
       catch (Exception ex) 
       { 
        _logger.LogError(ex.Message); 
        transaction.Rollback(); 
        connection.Close(); 
       } 
      } 

      transaction.Commit(); 
     } 
    } 

    #endregion 

    #region Private Helper Methods 

    /// <summary> 
    /// Creates the custom data table. 
    /// </summary> 
    private DataTable CreateCustomDataTable() 
    { 
     PropertyDescriptorCollection properties = TypeDescriptor.GetProperties(typeof(T)); 
     var table = new DataTable(); 
     foreach (PropertyDescriptor prop in properties) 
     { 
      // Just include the not excluded columns 
      if (_excludedPropertyNames.All(epn => epn != prop.Name)) 
      {     
       if (prop.PropertyType.Name == "DbGeography") 
       { 
        var type = typeof(SqlGeography); 
        table.Columns.Add(prop.Name, type); 
       } 
       else 
       { 
        table.Columns.Add(prop.Name, Nullable.GetUnderlyingType(prop.PropertyType) ?? prop.PropertyType); 
       } 
      } 
     } 
     foreach (T item in _data) 
     { 
      DataRow row = table.NewRow(); 
      foreach (PropertyDescriptor prop in properties) 
      { 
       // Just include the values in not excluded properties 
       if (_excludedPropertyNames.All(epn => epn != prop.Name)) 
       { 
        if (prop.PropertyType.Name == "DbGeography") 
        {       
         row[prop.Name] = SqlGeography.Parse(((DbGeography)prop.GetValue(item)).AsText()).MakeValid(); 
        } 
        else 
        { 
         row[prop.Name] = prop.GetValue(item) ?? DBNull.Value; 
        } 
       } 
      } 
      table.Rows.Add(row); 
     } 
     return table; 
    } 

    #endregion 

} 

它的使用會如:

//1st.- You would have a colection of entities: 
var myEntities = new List<MyEntity>(); 
// [...] With thousands or millions of items 

// 2nd.- You would create the BulkInsert: 
myEntityTypeBulk = new BulkInsert<MyEntity>(_connectionString, "MyEntitiesTableName", myEntities, new[] { "ObjectState", "SkippedEntityProperty1", "SkippedEntityProperty2" }); 

// 3rd.- You would execute it: 
myEntityTypeBulk.Insert(); 

獲得的性能和該類的可重用性值得此消息。我希望它能幫助:

娟您打算如何存放將是有益的,你可以用你自己的方法

+0

什麼是SqlGeography? –

+1

我應該評論說,當你在實體框架中使用地理空間數據時,解決了一個常見問題,類型不一樣,你的實體將使用DbGeography,EF知道並用於此目的的類型,但要插入它進入SQL它必須是SqlGeography。 – Juan