並行下載

我想通過在C＃中的HTTP實現文件的並行下載。我嘗試了幾種不同的方法，但他們都沒有正常工作。無論我做什麼，下載最終都會排隊等待，而不是以真正的並行方式工作。並行下載

任何人都可以給我一些指示或鏈接到所描述的實際工作方法的文章？

2011-07-01 phil

有什麼樣的方法你試過？請張貼一些代碼。 –

你到目前爲止嘗試過什麼？你是否啓動了兩個線程，並在每個線程中執行了Web請求，或者是否使用.NET 4的並行庫？展示你到目前爲止所做的努力。 –

缺乏細節使得這個問題很難回答。請發佈一些代碼，並讓我們知道，如果所有的下載來自相同/不同的域名。 – spender

我只是寫一些代碼，沒有測試，等待一些觀察感謝大家：

public class DownloadFile 
{ 
    public string Url { get; set; } 

    public string PathToSave { get; set; } 
} 


public class ParallelDownloading 
    { 
     private ConcurrentQueue<DownloadFile> _queueToDownlaod; 
     private IList<Task> _downloadingTasks; 
     private Timer _downloadTimer; 

     private int _parallelDownloads; 

     public ParallelDownloading(int parallelDownloads) 
     { 
      _queueToDownlaod = new ConcurrentQueue<DownloadFile>(); 
      _downloadingTasks = new List<Task>(); 
      _downloadTimer = new Timer(); 

      _parallelDownloads = parallelDownloads; 

      _downloadTimer.Elapsed += new ElapsedEventHandler(DownloadTimer_Elapsed); 
      _downloadTimer.Interval = 1000; 
      _downloadTimer.Start(); 

      ServicePointManager.DefaultConnectionLimit = parallelDownloads; 
     } 

     public void EnqueueFileToDownload(DownloadFile file) 
     { 
      _queueToDownlaod.Enqueue(file); 
     } 

     void DownloadTimer_Elapsed(object sender, ElapsedEventArgs e) 
     { 
      StartDownload(); 
     } 

     private void StartDownload() 
     { 
      lock (_downloadingTasks) 
      { 
       if (_downloadingTasks.Count < _parallelDownloads && _queueToDownlaod.Count > 0) 
       { 
        DownloadFile fileToDownload; 
        if (_queueToDownlaod.TryDequeue(out fileToDownload)) 
        { 
         var task = new Task(() => 
         { 
          var client = new WebClient(); 
          client.DownloadFile(fileToDownload.Url, fileToDownload.PathToSave); 
         }, TaskCreationOptions.LongRunning); 

         task.ContinueWith(DownloadOverCallback, TaskContinuationOptions.None); 

         _downloadingTasks.Add(task); 
         task.Start(); 
        }  
       } 
      } 
     } 

     public void DownloadOverCallback(Task downloadingTask) 
     { 
      lock (_downloadingTasks) 
      { 
       _downloadingTasks.Remove(downloadingTask); 
      } 
     } 
    }

你可以用這個測試：

ParallelDownloading p = new ParallelDownloading(5); 

     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file1.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file2.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file3.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file4.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file5.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file6.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file7.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file8.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file9.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file10.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file11.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file12.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file13.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file14.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file15.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file16.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file17.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file18.f", Url = @"http://download.thinkbroadband.com/20MB.zip" }); 
     p.EnqueueFileToDownload(new DownloadFile() { PathToSave = @"c:\file19.f", Url = @"http://download.thinkbroadband.com/20MB.zip" });

來源

2011-07-01 13:25:18 alexl