一次讀取一個文件的行數C＃

Question

我想每次讀取和處理10行以上的GB文件，但還沒有找到解決方案來吐出10行直到結束。

我最後的嘗試是：

 int n = 10; 
     foreach (var line in File.ReadLines("path") 
      .AsParallel().WithDegreeOfParallelism(n)) 
     { 
      System.Console.WriteLine(line); 
      Thread.Sleep(1000); 
     } 

我已經看到了使用緩衝區的大小的解決方案，但我想整個行來讀。

Answer 1

保重ReadLines將讀取您的GB文件的所有行，不僅是您打印的 打印的行。 你真的需要並行嗎？

Answer 2

默認behavour是讀取所有行在一個鏡頭，如果你想讀取少於，你需要深入瞭解它如何讀取它們，並得到一個StreamReader然後讓你控制讀取過程

 using (StreamReader sr = new StreamReader(path)) 
     { 
      while (sr.Peek() >= 0) 
      { 
       Console.WriteLine(sr.ReadLine()); 
      } 
     } 

它也有一個ReadLineAsync方法將返回一個任務

如果包含在ConcurrentBag這些任務，你可以很容易保持加工上10日線同時運行。

var bag =new ConCurrentBag<Task>(); 
using (StreamReader sr = new StreamReader(path)) 
{ 
    while(sr.Peek() >=0) 
    { 
     if(bag.Count < 10) 
     { 
      Task processing = sr.ReadLineAsync().ContinueWith((read) => { 
       string s = read.Result;//EDIT Removed await to reflect Scots comment 
       //process line 
      }); 
      bag.Add(processing); 
     } 
     else 
     { 
      Task.WaitAny(bag.ToArray()) 
      //remove competed tasks from bag 
     } 
    } 
} 

注意此代碼僅供參考，不得作爲原樣使用;

如果你想要的是最後十行，那麼你可以得到與這裏 How to read a text file reversely with iterator in C#

Answer 3

解決此方法將創建一個從您的文件中的行「頁面」。

public static IEnumerable<string[]> ReadFileAsLinesSets(string fileName, int setLen = 10) 
{ 
    using (var reader = new StreamReader(fileName)) 
     while (!reader.EndOfStream) 
     { 
      var set = new List<string>(); 
      for (var i = 0; i < setLen && reader.EndOfStream; i++) 
      { 
       set.Add(reader.ReadLine()); 
      } 
      yield return set.ToArray(); 
     } 
} 

...更多好玩的版本...

class Example 
{ 
    static void Main(string[] args) 
    { 
     "YourFile.txt".ReadAsLines() 
         .AsPaged(10) 
         .Select(a=>a.ToArray()) //required or else you will get random data since "WrappedEnumerator" is not thread safe 
         .AsParallel() 
         .WithDegreeOfParallelism(10) 
         .ForAll(a => 
     { 
      //Do your work here. 
      Console.WriteLine(a.Aggregate(new StringBuilder(), 
              (sb, v) => sb.AppendFormat("{0:000000} ", v), 
              sb => sb.ToString())); 
     }); 
    } 
} 

public static class ToolsEx 
{ 

    public static IEnumerable<IEnumerable<T>> AsPaged<T>(this IEnumerable<T> items, 
                   int pageLength = 10) 
    { 
     using (var enumerator = new WrappedEnumerator<T>(items.GetEnumerator())) 
      while (!enumerator.IsDone) 
       yield return enumerator.GetNextPage(pageLength); 
    } 

    public static IEnumerable<T> GetNextPage<T>(this IEnumerator<T> enumerator, 
                int pageLength = 10) 
    { 
     for (var i = 0; i < pageLength && enumerator.MoveNext(); i++) 
      yield return enumerator.Current; 
    } 

    public static IEnumerable<string> ReadAsLines(this string fileName) 
    { 
     using (var reader = new StreamReader(fileName)) 
      while (!reader.EndOfStream) 
       yield return reader.ReadLine(); 
    } 
} 

internal class WrappedEnumerator<T> : IEnumerator<T> 
{ 
    public WrappedEnumerator(IEnumerator<T> enumerator) 
    { 
     this.InnerEnumerator = enumerator; 
     this.IsDone = false; 
    } 

    public IEnumerator<T> InnerEnumerator { get; private set; } 
    public bool IsDone { get; private set; } 

    public T Current { get { return this.InnerEnumerator.Current; } } 
    object System.Collections.IEnumerator.Current { get { return this.Current; } } 

    public void Dispose() 
    { 
     this.InnerEnumerator.Dispose(); 
     this.IsDone = true; 
    } 

    public bool MoveNext() 
    { 
     var next = this.InnerEnumerator.MoveNext(); 
     this.IsDone = !next; 
     return next; 
    } 

    public void Reset() 
    { 
     this.IsDone = false; 
     this.InnerEnumerator.Reset(); 
    } 
}