將一百萬行寫入文本文件

Question

請參閱以下用於寫入文本文件的代碼。但是，數據庫獲取近1百萬條記錄，所以有人可以建議一個更快的方式來做到這一點，或者我應該如何改變下面提到的代碼，以使其工作更快？

try 
{ 
    using (OleDbConnection connection = new OleDbConnection(ConnectionString)) 
    { 
     OleDbCommand command = new OleDbCommand(queryString, connection); 
     connection.Open(); 
     OleDbDataReader reader = command.ExecuteReader(); 
     DataTable dt = new DataTable(); 
     dt.Load(reader); 

     StreamWriter writer = new StreamWriter(FilePath + FileName); 
     var result = string.Empty; 
     for (int i = 0; i < dt.Rows.Count; i++) 
     { 
      for (int j = 0; j < dt.Columns.Count; j++) 
      { 
       result += dt.Rows[i][j] + "|"; 
      } 
      result += "\r\n"; 
     } 
     writer.WriteLine(result); 

     reader.Close(); 
     writer.Close(); 

     Dts.TaskResult = (int)ScriptResults.Success; 
    } 
} 

Answer 1

您在一個數據表中加載100萬條記錄，然後再次在該表上循環寫入每一行。

也許你可以做同樣的事情，但在一半的時間，如果你讀了單排，並寫下來：也

OleDbDataReader reader = command.ExecuteReader(); 
StreamWriter writer = new StreamWriter(FilePath + FileName); 
var result = string.Empty; 
while(reader.Read()) 
{ 
    for (int j = 0; j < reader.FieldCount; j++) 
    { 
     result += reader[j] + "|"; 
    } 
    result += "\r\n"; 
} 
writer.WriteLine(result); 

，使用StringBuilder的緩衝你讀的內存，而不是不斷地寫磁盤可能是在這種情況下是非常有益的：

// This size is just for example purpose. Should be fine tuned 
StringBuilder buffer = new StringBuilder(1048576); 
while(reader.Read()) 
{ 
    for (int j = 0; j < reader.FieldCount; j++) 
    { 
     buffer.Append(reader[j] + "|"); 
    } 
    buffer.AppendLine(); 
    if(buffer.Length > 1048576 - 1024) 
    { 
     writer.Write(buffer.ToString()); 
     buffer.Length = 0; 
    } 
} 
writer.Write(buffer.ToString());