SQLite插入：內存：性能

Question

我正在尋找一個使用SQLite內存表的「緩存」解決方案。 快速插入和快速選擇是高優先級。

的問題是，我不明白的插入速度我在SQLite的期待。

我標杆SQLite的內存中的表與做BulkCopyInserts到SQL Server數據庫。我可以以50 000記錄/秒的速度將100 000條記錄BulkCopy放入表中，但使用SQLite我的速度爲12 500記錄/秒。

任何想法，我能做些什麼來提高性能？ 是否沒有辦法將數據推送到表中而沒有循環併爲每條記錄進行插入操作？

/埃裏克

 private void SQLLiteRunButton_Click(object sender, EventArgs e) 
    { 

     int noOfRecords = Convert.ToInt32(NoOfRecordsTextBox.Text); 

     SQLLiteListBox.Items.Add("Start generating "+noOfRecords+" in memory"); 
     DataTable table = GetTable(noOfRecords); 

     var dbConnection = new SQLiteConnection("Data Source=:memory:;Version = 3;"); 

     dbConnection.Open(); 

     //Create Target Table 
     string sql = "CREATE TABLE LedgerSumTemp([RowID] [int] NOT NULL,[Client_ID] [nvarchar](50) NULL,[ISOMONTH] [nvarchar](50) NULL,[Date] [datetime] NULL, [Dim1] [nvarchar](50) NULL,[Dim2] [nvarchar](50) NULL, [Dim3] [nvarchar](50) NULL, [Dim4] [nvarchar](50) NULL, [Amount] [decimal](18, 6) NULL)"; 

     SQLiteCommand command = new SQLiteCommand(sql, dbConnection); 
     command.ExecuteNonQuery(); 

     //Set insertTimeStart 
     Stopwatch stopwatch = new Stopwatch(); 
     stopwatch.Start(); 

     //Fill table with data fast - Alternative 2 – using params 
     using (var cmd = new SQLiteCommand(dbConnection)) 
     { 


      using (var transaction = dbConnection.BeginTransaction()) 
      { 

       cmd.CommandText = "INSERT INTO LedgerSumTemp (RowID, Client_ID, ISOMONTH, Date, Dim1, Dim2, Dim3, Dim4, Amount) VALUES (@RowID,@Client_ID,@ISOMONTH,@Date,@Dim1,@Dim2,@Dim3,@Dim4,@Amount);"; 
       cmd.CommandType = CommandType.Text; 

       foreach (DataRow row in table.Rows) 
       { 
        cmd.Parameters.Add(new SQLiteParameter("@RowID", row["RowID"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Client_ID", row["Client_ID"])); 
        cmd.Parameters.Add(new SQLiteParameter("@ISOMONTH", row["ISOMONTH"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Date", row["Date"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Dim1", row["Dim1"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Dim2", row["Dim2"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Dim3", row["Dim3"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Dim4", row["Dim4"])); 
        cmd.Parameters.Add(new SQLiteParameter("@Amount", row["Amount"])); 

        cmd.ExecuteNonQuery(); 

       } 

       transaction.Commit(); 
      } 
     } 



     stopwatch.Start(); 
     int mS = Convert.ToInt32(stopwatch.Elapsed.TotalMilliseconds); 

     double s = mS/1000; 
     double recPerSec = (noOfRecords/s); 

     SQLLiteListBox.Items.Add(noOfRecords.ToString()+" Time: " + mS.ToString()+ " mSec"+" Speed: "+recPerSec.ToString() +" RecPerSec"); 
     SQLLiteListBox.Items.Add("--------END-------"); 

}

static DataTable GetTable(int NoOfRows) 
    { 

     int noOfRows = NoOfRows; 

     //Create DataTable in memory 
     DataTable table = new DataTable(); 
     table.TableName = "LedgerSumTemp"; 
     table.Columns.Add("RowID", typeof(int)); 
     table.Columns.Add("Client_ID", typeof(string)); 
     table.Columns.Add("ISOMONTH", typeof(string)); 
     table.Columns.Add("Date", typeof(DateTime)); 
     table.Columns.Add("Dim1", typeof(string)); 
     table.Columns.Add("Dim2", typeof(string)); 
     table.Columns.Add("Dim3", typeof(string)); 
     table.Columns.Add("Dim4", typeof(string)); 
     table.Columns.Add("Amount", typeof(double)); 

     //Add rows 
     for (int i = 0; i < noOfRows; i++) 
     { 
      table.Rows.Add(i, "10", "2013"+i.ToString(), DateTime.Now, "3010", "110", "A58", "1000", Convert.ToDouble(i)); 
     } 

     return table; 
    } 

Answer 1

您的代碼創建在每個循環迭代新的參數對象。

您應該重新使用他們，只是改變他們的價值觀;像這樣：

... 
var paramRowID = cmd.CreateParameter(); 
paramRowID.ParameterName = "@RowID"; 
cmd.Parameters.Add(paramRowID); 
// same for the other parameters 

foreach ... 
{ 
    paramRowID.Value = row["RowID"]; 
    // same for the other parameters 
    cmd.ExecuteNonQuery(); 
} 

但是，SQLite沒有類似於BulkCopy的機制;你總是必須逐個插入記錄。