寫入文件內存不足c＃

Question

我得到了一些與C＃窗體的問題。
我的目標是將一個大文件（可能> 5GB）分割成文件，每個文件包含一百萬行。
根據下面的代碼，我不知道爲什麼它會出現內存不足。

謝謝。

StreamReader readfile = new StreamReader(...); 
StreamWriter writefile = new StreamWriter(...);  
string content; 
while ((content = readfile.ReadLine()) != null) 
{ 
    writefile.Write(content + "\r\n"); 
    i++; 
    if (i % 1000000 == 0) 
    { 
     index++; 
     writefile.Close(); 
     writefile.Dispose(); 
     writefile = new StreamWriter(...); 
    } 
    label5.Text = i.ToString(); 
    label5.Update(); 
} 

Answer 1

該錯誤可能是在

label5.Text = i.ToString(); 
label5.Update(); 

只是爲了測試我寫的是這樣的：

for (int i = 0; i < int.MaxValue; i++) 
{ 
    label1.Text = i.ToString(); 
    label1.Update(); 
} 

該應用程序凍結圍繞16000-18000（Windows 7的臨SP1 x64，運行x86和x64的應用程序）。

可能發生的情況是，通過在應用程序的主線程中運行長操作，可以停止窗口的消息隊列，並在某個點凍結。你可以看到，這是通過添加

Application.DoEvents(); 

，而不是

label5.Update(); 

問題，但即使這是一個錯誤的解決方案。正確的解決方案是移動複製在另一個線程，並更新每x毫秒的控制下，使用Invoke方法（因爲你是在輔助線程），

例如：

public void Copy(string source, string dest) 
{ 
    const int updateMilliseconds = 100; 

    int index = 0; 
    int i = 0; 
    StreamWriter writefile = null; 

    try 
    { 
     using (StreamReader readfile = new StreamReader(source)) 
     { 
      writefile = new StreamWriter(dest + index); 

      // Initial value "back in time". Forces initial update 
      int milliseconds = unchecked(Environment.TickCount - updateMilliseconds); 

      string content; 
      while ((content = readfile.ReadLine()) != null) 
      { 
       writefile.Write(content); 
       writefile.Write("\r\n"); // Splitted to remove a string concatenation 
       i++; 

       if (i % 1000000 == 0) 
       { 
        index++; 
        writefile.Dispose(); 
        writefile = new StreamWriter(dest + index); 

        // Force update 
        milliseconds = unchecked(milliseconds - updateMilliseconds); 
       } 

       int milliseconds2 = Environment.TickCount; 

       int diff = unchecked(milliseconds2 - milliseconds); 

       if (diff >= updateMilliseconds) 
       { 
        milliseconds = milliseconds2; 
        Invoke((Action)(() => label5.Text = string.Format("File {0}, line {1}", index, i))); 
       } 
      } 
     } 
    } 
    finally 
    { 
     if (writefile != null) 
     { 
      writefile.Dispose(); 
     } 
    } 

    // Last update 
    Invoke((Action)(() => label5.Text = string.Format("File {0}, line {1} Finished", index, i))); 
} 

並用稱之爲：

var thread = new Thread(() => Copy(@"C:\Temp\lst.txt", @"C:\Temp\output")); 
thread.Start(); 

說明它將如何寫label5每100毫秒，再加上一旦開頭（通過設置milliseconds「時光倒流」的初始值），每次輸出文件的變化d（通過設置milliseconds的值「回到過去」）以及處理完所有內容後。

通過使用BackgroundWorker類可以編寫一個更正確的示例，該類明確存在於此場景中。它有一個事件，ProgressChanged，可以訂閱更新窗口。

事情是這樣的：

private void button1_Click(object sender, EventArgs e) 
{ 
    BackgroundWorker backgroundWorker = new BackgroundWorker(); 
    backgroundWorker.WorkerReportsProgress = true; 
    backgroundWorker.ProgressChanged += backgroundWorker_ProgressChanged; 
    backgroundWorker.RunWorkerCompleted += backgroundWorker_RunWorkerCompleted; 
    backgroundWorker.DoWork += backgroundWorker_DoWork; 
    backgroundWorker.RunWorkerAsync(new string[] { @"C:\Temp\lst.txt", @"C:\Temp\output" }); 
} 

private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e) 
{ 
    BackgroundWorker worker = sender as BackgroundWorker; 

    string[] arguments = (string[])e.Argument; 
    string source = arguments[0]; 
    string dest = arguments[1]; 

    const int updateMilliseconds = 100; 

    int index = 0; 
    int i = 0; 
    StreamWriter writefile = null; 

    try 
    { 
     using (StreamReader readfile = new StreamReader(source)) 
     { 
      writefile = new StreamWriter(dest + index); 

      // Initial value "back in time". Forces initial update 
      int milliseconds = unchecked(Environment.TickCount - updateMilliseconds); 

      string content; 
      while ((content = readfile.ReadLine()) != null) 
      { 
       writefile.Write(content); 
       writefile.Write("\r\n"); // Splitted to remove a string concatenation 
       i++; 

       if (i % 1000000 == 0) 
       { 
        index++; 
        writefile.Dispose(); 
        writefile = new StreamWriter(dest + index); 

        // Force update 
        milliseconds = unchecked(milliseconds - updateMilliseconds); 
       } 

       int milliseconds2 = Environment.TickCount; 

       int diff = unchecked(milliseconds2 - milliseconds); 

       if (diff >= updateMilliseconds) 
       { 
        milliseconds = milliseconds2; 
        worker.ReportProgress(0, new int[] { index, i }); 
       } 
      } 
     } 
    } 
    finally 
    { 
     if (writefile != null) 
     { 
      writefile.Dispose(); 
     } 
    } 

    // For the RunWorkerCompleted 
    e.Result = new int[] { index, i }; 
} 

void backgroundWorker_ProgressChanged(object sender, ProgressChangedEventArgs e) 
{ 
    int[] state = (int[])e.UserState; 
    label5.Text = string.Format("File {0}, line {1}", state[0], state[1]); 
} 

void backgroundWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e) 
{ 
    int[] state = (int[])e.Result; 
    label5.Text = string.Format("File {0}, line {1} Finished", state[0], state[1]); 
}