MySql插入忽略性能

Question

我必須將90Mb的數據插入到MySql表中，並且我正在使用INSERT IGNORE命令來避免重複鍵的異常。表演每秒8記錄，但看起來非常緩慢。我可以加快速度嗎？

p.s.我將每個記錄的記錄，因爲我從SQL壓縮數據庫

using (SqlCeConnection sqlConnection = new SqlCeConnection(connectionstrCe)) 
      { 
       sqlConnection.Open(); 
       SqlCeCommand cmdCe = sqlConnection.CreateCommand(); 
       { 
        { 

         mySQLConnection.Open(); 

         foreach (KeyValuePair<string, List<string>> t in tablesCeNames) //reading the tables property from the dictionary - column names and datatypes 
         { 

          string tableData = t.Key; 
          List<string> columnData = t.Value; 
//get the values from the table I want to transfer the data 
           cmdText = "SELECT * FROM " + tableData; 
//compose the mysql command 
           cmdCe.CommandText = cmdText; 

           SqlCeDataReader dataReader = cmdCe.ExecuteReader(); //read 
//InsertTable is a method that get the datareader and convert all the data from this table in a list array with the values to insert 
           inputValues = InsertTables(dataReader); 


           MySql.Data.MySqlClient.MySqlTransaction transakcija; 
           transakcija = mySQLConnection.BeginTransaction(); 

           worker.ReportProgress(4, inputValues.Count); 

           foreach (string val in inputValues)//foreach row of values of the data table 
           { 
            cmdSqlText = "INSERT IGNORE INTO " + tableData + "("; //compose the command for sql 

            foreach (string cName in columnData) //forach column in a table 
            { 
             string[] data = cName.Split(' '); 
             if (!data[0].ToString().Equals("Id")) 
             { 
              cmdSqlText += data[0].ToString() + ","; //write the column names of the values that will be inserted 
             } 
            } 
            cmdSqlText = cmdSqlText.TrimEnd(','); 
            cmdSqlText += ") VALUES ("; 
//val contains the values of this current record that i want to insert 
            cmdSqlText += val; //final command with insert ignore and the values of one record 
            if (!val.Equals("")) 
            { 
             try 
             { 
              new MySql.Data.MySqlClient.MySqlCommand(cmdSqlText, mySQLConnection, transakcija).ExecuteNonQuery(); //execute insert on sql database 
              WriteToTxt("uspješno upisano u mysql" + t.Key); 
             } 
             catch (MySql.Data.MySqlClient.MySqlException sqlEx) 
             { 
             } 
            } 
           } 

           if (TablicaSveOK) 
           { 
            transakcija.Commit(); 

           } 
           else 
           { 
            transakcija.Rollback(); 
           } 
          } 
         } 

         if (mySQLConnection.State != System.Data.ConnectionState.Closed) 
         { 
          mySQLConnection.Close(); 
         } 
        } 

Answer 1

什麼讓從SQL數據到一個文件中，並使用LOAD DATA讀取數據？

http://dev.mysql.com/doc/refman/5.0/es/load-data.html

Answer 2

而是然後發送多個電話，您可以發送一個呼叫插入的所有記錄。從這個插入像這樣

insert into your table(col1, col2) 
SELECT 'Name1', 'Location1' 
UNION ALL 
SELECT 'Name2', 'Location2' 
UNION ALL 
SELECT 'Name3', 'Location3' 

的一部分有可能是你的代碼是瓶頸，而不是INSERT語句。所以我建議你首先檢查問題所在，然後尋求解決方案。

Answer 3

最新的MySql.Connector有一個名爲MySqlBulkLoader的類，可以用來以更加面向.NET的方式調用MySql的LOAD DATA語法。該類需要一個逗號分隔值文件來加載，並且速度非常快。

因此，您的工作是讀取數據表中的所有Sql Compact記錄，然後使用Streamwriter或專用的CSV Writer寫下文件中的所有內容。

然後將代碼加載在一個MySQL表數據是簡單的像這樣

string connStr = "server=localhost;user=root;database=........"; 
using(MySqlConnection conn = new MySqlConnection(connStr)) 
{ 
    MySqlBulkLoader bl = new MySqlBulkLoader(conn); 
    bl.TableName = "yourdestinationtable"; 
    bl.FieldTerminator = "\t"; 
    bl.LineTerminator = "\n"; 
    bl.FileName = "path_to_your_comma_separated_value_file"; 
    try 
    { 
     conn.Open(); 
     int count = bl.Load(); 
    } 
}