異步和並行下載文件

Question

編輯

我已經改變了問題的標題來反映這個問題我有，但是還就如何輕鬆地實現這個答案。

---

我試圖讓第二個方法返回Task<TResult>而不是Task作爲第一個方法，但我得到錯誤的級聯爲試圖解決它的後果。

• 我await body(partition.Current);
• 前加入return反過來要求我先下添加，所以我下面
• 添加return null return語句但是現在的select語句抱怨說，它不能從查詢
推斷類型參數
• 我將Task.Run更改爲Task.Run<TResult>，但沒有成功。

我怎樣才能解決這個問題？

第一種方法來自http://blogs.msdn.com/b/pfxteam/archive/2012/03/05/10278165.aspx，第二種方法是，我試圖創造過載。

public static class Extensions 
{ 
    public static Task ForEachAsync<T>(this IEnumerable<T> source, int dop, Func<T, Task> body) 
    { 
     return Task.WhenAll(
      from partition in Partitioner.Create(source).GetPartitions(dop) 
      select Task.Run(async delegate 
      { 
       using (partition) 
        while (partition.MoveNext()) 
         await body(partition.Current); 
      })); 
    } 

    public static Task ForEachAsync<T, TResult>(this IEnumerable<T> source, int dop, Func<T, Task<TResult>> body) 
    { 
     return Task.WhenAll(
      from partition in Partitioner.Create(source).GetPartitions(dop) 
      select Task.Run(async delegate 
      { 
       using (partition) 
        while (partition.MoveNext()) 
         await body(partition.Current); 
      })); 
    } 
} 

用例：

使用這種方法，我想下載並行且異步多個文件：

private async void MainWindow_Loaded(object sender, RoutedEventArgs e) 
{ 
    Artist artist = await GetArtist(); 
    IEnumerable<string> enumerable = artist.Reviews.Select(s => s.ImageUrl); 
    string[] downloadFile = await DownloadFiles(enumerable); 
} 

public static async Task<string[]> DownloadFiles(IEnumerable<string> enumerable) 
{ 
    if (enumerable == null) throw new ArgumentNullException("enumerable"); 
    await enumerable.ForEachAsync(5, s => DownloadFile(s)); 
    // Incomplete, the above statement is void and can't be returned 
} 

public static async Task<string> DownloadFile(string address) 
{ 
    /* Download a file from specified address, 
     * return destination file name on success or null on failure */ 

    if (address == null) 
    { 
     return null; 
    } 

    Uri result; 
    if (!Uri.TryCreate(address, UriKind.Absolute, out result)) 
    { 
     Debug.WriteLine(string.Format("Couldn't create URI from specified address: {0}", address)); 
     return null; 
    } 

    try 
    { 
     using (var client = new WebClient()) 
     { 
      string fileName = Path.GetTempFileName(); 
      await client.DownloadFileTaskAsync(address, fileName); 
      Debug.WriteLine(string.Format("Downloaded file saved to: {0} ({1})", fileName, address)); 
      return fileName; 
     } 
    } 
    catch (WebException webException) 
    { 
     Debug.WriteLine(string.Format("Couldn't download file from specified address: {0}", webException.Message)); 
     return null; 
    } 
} 

Answer 1

我解決了它，並在這裏張貼，可以幫助具有人同樣的問題。

我最初的需求是一個小幫手，它可以快速下載圖像，但如果服務器沒有快速響應，只是丟棄連接，所有這些並行和異步。

此幫助程序將返回包含遠程路徑，本地路徑和發生異常的元組;非常有用，因爲它總是很好的知道爲什麼錯誤的下載出現故障。我想我沒有忘記下載時可能發生的任何情況，但歡迎您發表評論。

• 您指定的URL列表，下載
• 您可以指定將被保存在本地的文件名，如果沒有一個會爲你可選擇方便
• 取消所下載的持續時間（生成緩慢或無法訪問服務器）

你可以只使用DownloadFileTaskAsync本身或使用ForEachAsync助手並行異步下載。

代碼對如何使用它的一個例子：

private async void MainWindow_Loaded(object sender, RoutedEventArgs e) 
{ 
    IEnumerable<string> enumerable = your urls here; 
    var results = new List<Tuple<string, string, Exception>>(); 
    await enumerable.ForEachAsync(s => DownloadFileTaskAsync(s, null, 1000), (url, t) => results.Add(t)); 
} 

/// <summary> 
///  Downloads a file from a specified Internet address. 
/// </summary> 
/// <param name="remotePath">Internet address of the file to download.</param> 
/// <param name="localPath"> 
///  Local file name where to store the content of the download, if null a temporary file name will 
///  be generated. 
/// </param> 
/// <param name="timeOut">Duration in miliseconds before cancelling the operation.</param> 
/// <returns>A tuple containing the remote path, the local path and an exception if one occurred.</returns> 
private static async Task<Tuple<string, string, Exception>> DownloadFileTaskAsync(string remotePath, 
    string localPath = null, int timeOut = 3000) 
{ 
    try 
    { 
     if (remotePath == null) 
     { 
      Debug.WriteLine("DownloadFileTaskAsync (null remote path): skipping"); 
      throw new ArgumentNullException("remotePath"); 
     } 

     if (localPath == null) 
     { 
      Debug.WriteLine(
       string.Format(
        "DownloadFileTaskAsync (null local path): generating a temporary file name for {0}", 
        remotePath)); 
      localPath = Path.GetTempFileName(); 
     } 

     using (var client = new WebClient()) 
     { 
      TimerCallback timerCallback = c => 
      { 
       var webClient = (WebClient) c; 
       if (!webClient.IsBusy) return; 
       webClient.CancelAsync(); 
       Debug.WriteLine(string.Format("DownloadFileTaskAsync (time out due): {0}", remotePath)); 
      }; 
      using (var timer = new Timer(timerCallback, client, timeOut, Timeout.Infinite)) 
      { 
       await client.DownloadFileTaskAsync(remotePath, localPath); 
      } 
      Debug.WriteLine(string.Format("DownloadFileTaskAsync (downloaded): {0}", remotePath)); 
      return new Tuple<string, string, Exception>(remotePath, localPath, null); 
     } 
    } 
    catch (Exception ex) 
    { 
     return new Tuple<string, string, Exception>(remotePath, null, ex); 
    } 
} 

public static class Extensions 
{ 
    public static Task ForEachAsync<TSource, TResult>(
     this IEnumerable<TSource> source, 
     Func<TSource, Task<TResult>> taskSelector, Action<TSource, TResult> resultProcessor) 
    { 
     var oneAtATime = new SemaphoreSlim(5, 10); 
     return Task.WhenAll(
      from item in source 
      select ProcessAsync(item, taskSelector, resultProcessor, oneAtATime)); 
    } 

    private static async Task ProcessAsync<TSource, TResult>(
     TSource item, 
     Func<TSource, Task<TResult>> taskSelector, Action<TSource, TResult> resultProcessor, 
     SemaphoreSlim oneAtATime) 
    { 
     TResult result = await taskSelector(item); 
     await oneAtATime.WaitAsync(); 
     try 
     { 
      resultProcessor(item, result); 
     } 
     finally 
     { 
      oneAtATime.Release(); 
     } 
    } 
} 

我沒有改變的ForEachAsync簽名來選擇並行的水平，我會讓你的願望，你調整。

輸出例如：

DownloadFileTaskAsync (null local path): generating a temporary file name for http://cache.thephoenix.com/secure/uploadedImages/The_Phoenix/Music/CD_Review/main_OTR_Britney480.jpg 
DownloadFileTaskAsync (null local path): generating a temporary file name for http://ssimg.soundspike.com/artists/britneyspears_femmefatale_cd.jpg 
DownloadFileTaskAsync (null local path): generating a temporary file name for http://a323.yahoofs.com/ymg/albumreviewsuk__1/albumreviewsuk-526650850-1301400550.jpg?ymm_1xEDE5bu0tMi 
DownloadFileTaskAsync (null remote path): skipping 
DownloadFileTaskAsync (time out due): http://hangout.altsounds.com/geek/gars/images/3/9/8/5/2375.jpg 
DownloadFileTaskAsync (time out due): http://www.beat.com.au/sites/default/files/imagecache/630_315sr/images/article/header/2011/april/britney-spears-femme-fatale.jpg 
DownloadFileTaskAsync (time out due): http://cache.thephoenix.com/secure/uploadedImages/The_Phoenix/Music/CD_Review/main_OTR_Britney480.jpg 
DownloadFileTaskAsync (downloaded): http://newblog.thecmuwebsite.com/wp-content/uploads/2009/12/britneyspears1.jpg 
DownloadFileTaskAsync (downloaded): http://newblog.thecmuwebsite.com/wp-content/uploads/2009/12/britneyspears1.jpg 
DownloadFileTaskAsync (downloaded): http://static.guim.co.uk/sys-images/Music/Pix/site_furniture/2011/3/22/1300816812640/Femme-Fatale.jpg 
DownloadFileTaskAsync (downloaded): http://www.sputnikmusic.com/images/albums/72328.jpg 

過去需要1分鐘的時間，現在僅僅需要10秒，同樣的結果:)

而且非常感謝這2個職位作者：

http://blogs.msdn.com/b/pfxteam/archive/2012/03/05/10278165.aspx

http://blogs.msdn.com/b/pfxteam/archive/2012/03/04/10277325.aspx