如何跟蹤長時間運行的並行程序的運行位置

Question

我有一個可重新啓動的程序，它運行在一個非常大的空間上，我已經開始對它進行一些並行化。每個任務獨立運行並用其結果更新數據庫。重複任務並不重要（它們基於輸入數組是完全確定的，並且會產生與之前相同的結果），但這樣做效率相對較低。到目前爲止，我想出了以下模式：

static void Main(string[] args) { 
     GeneratorStart = Storage.Load(); 
     var tasks = new List<Task>(); 
     foreach (int[] temp in Generator()) { 
      var arr = temp; 
      var task = new Task(() => { 
       //... use arr as needed 
      }); 
      task.Start(); 
      tasks.Add(task); 
      if (tasks.Count > 4) { 
       Task.WaitAll(tasks.ToArray()); 
       Storage.UpdateStart(temp); 
       tasks = new List<Task>(); 
      } 
     } 
    } 

之前使發電機重新啓動，我就可以了簡單Parallel.Foreach循環，是一個快一點。我想我正在用WaitAll操作丟失一些CPU時間。我怎樣才能擺脫這個瓶頸，同時跟蹤什麼任務，我不必再次運行時，我重新啓動？

對於那些關注其他位（縮短爲簡潔質疑）：

class Program { 
    static bool Done = false; 
    static int[] GeneratorStart = null; 
    static IEnumerable<int[]> Generator() { 
     var s = new Stack<int>(); 
     //... omitted code to initialize stack to GeneratorStart for brevity 
     yield return s.ToArray(); 
     while (!Done) { 
      Increment(s); 
      yield return s.Reverse().ToArray(); 
     } 
    } 

    static int Base = 25600; //example number (none of this is important 
    static void Increment(Stack<int> stack) { //outside the fact 
     if (stack.Count == 0) {    //that it is generating an array 
      stack.Push(1);     //of a large base 
      return;       //behaving like an integer 
     }          //with each digit stored in an 
     int i = stack.Pop();     //array position) 
     i++; 
     if (i < Base) { 
      stack.Push(i); 
      return; 
     } 
     Increment(stack); 
     stack.Push(0); 
    } 
} 

Answer 1

我想出這個：

var tasks = new Queue<Pair<int[],Task>>(); 
    foreach (var temp in Generator()) { 
     var arr = temp; 
     tasks.Enqueue(new Pair<int[], Task>(arr, Task.Run(() ={ 
      //... use arr as needed 
     })); 
     var tArray = t.Select(v => v.Value).Where(t=>!t.IsCompleted).ToArray(); 
     if (tArray.Length > 7) { 
      Task.WaitAny(tArray); 
      var first = tasks.Peek(); 
      while (first != null && first.B.IsCompleted) { 
       Storage.UpdateStart(first.A); 
       tasks.Dequeue(); 
       first = tasks.Count == 0 ? null : tasks.Peek(); 
      } 
     } 
    } 

... 
class Pair<TA,TB> { 
    public TA A { get; set; } 
    public TB B { get; set; } 
    public Pair(TA a, TB b) { A = a; B = b; } 
}