通過慢速網絡下載大文件時的StorageException

Question

我正在使用NuGet程序包WindowsAzure.Storage版本4.2.1。

以下代碼嘗試從位於遙遠的數據中心的存儲容器中下載blob。

try 
{ 
    var blobRequestOptions = new BlobRequestOptions 
    { 
     RetryPolicy = new ExponentialRetry(TimeSpan.FromSeconds(5), 3), 
     MaximumExecutionTime = TimeSpan.FromMinutes(60), 
     ServerTimeout = TimeSpan.FromMinutes(60) 
    }; 
    using (var fileStream = File.Create(localPath)) 
    { 
     blockBlob.DownloadToStream(fileStream, null, blobRequestOptions); 
    } 
} 
catch (Exception e) 
{ 
    Console.WriteLine(e.Message); 
} 

但是，有時它下載了約10分鐘，然後它拋出以下異常：

Unhandled Exception: Microsoft.WindowsAzure.Storage.StorageException: The client could not finish the operation within specified timeout. ---> System.TimeoutException: The client could not finish the operation within specified timeout. 
--- End of inner exception stack trace --- 
at Microsoft.WindowsAzure.Storage.Core.Util.StorageAsyncResult`1.End() 
at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.EndUploadText(IAsyncResult asyncResult) 
at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass4.b__3(IAsyncResult ar) 
--- End of stack trace from previous location where exception was thrown --- 

我該如何解決這個問題？

Answer 1

請試試這段代碼。基本上它所做的是首先創建一個空文件，然後使用DownloadRangeToStream方法以1 MB塊讀取blob的數據。隨着塊被下載，它會將其附加到文件中。

private static void DownloadLargeFile() 
    { 
     var account = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true); 
     var blobClient = account.CreateCloudBlobClient(); 
     var container = blobClient.GetContainerReference("container-name"); 
     var file = "my-very-large-file-name"; 
     var blob = container.GetBlockBlobReference(file); 
     //First fetch the size of the blob. We use this to create an empty file with size = blob's size 
     blob.FetchAttributes(); 
     var blobSize = blob.Properties.Length; 
     long blockSize = (1 * 1024 * 1024);//1 MB chunk; 
     blockSize = Math.Min(blobSize, blockSize); 
     //Create an empty file of blob size 
     using (FileStream fs = new FileStream(file, FileMode.Create))//Create empty file. 
     { 
      fs.SetLength(blobSize);//Set its size 
     } 
     var blobRequestOptions = new BlobRequestOptions 
     { 
      RetryPolicy = new ExponentialRetry(TimeSpan.FromSeconds(5), 3), 
      MaximumExecutionTime = TimeSpan.FromMinutes(60), 
      ServerTimeout = TimeSpan.FromMinutes(60) 
     }; 
     long currentPointer = 0; 
     long bytesRemaining = blobSize; 
     do 
     { 
      var bytesToFetch = Math.Min(blockSize, bytesRemaining); 
      using (MemoryStream ms = new MemoryStream()) 
      { 
       //Download range (by default 1 MB) 
       blob.DownloadRangeToStream(ms, currentPointer, bytesToFetch, null, blobRequestOptions); 
       ms.Position = 0; 
       var contents = ms.ToArray(); 
       using (var fs = new FileStream(file, FileMode.Open))//Open that file 
       { 
        fs.Position = currentPointer;//Move the cursor to the end of file. 
        fs.Write(contents, 0, contents.Length);//Write the contents to the end of file. 
       } 
       currentPointer += contents.Length;//Update pointer 
       bytesRemaining -= contents.Length;//Update bytes to fetch 
      } 
     } 
     while (bytesRemaining > 0); 
    } 

我試過這個代碼在一個小文件上（可憐的互聯網連接：P）。因此，請在嘗試使用500 MB文件之前，先在一個小文件（比如5 - 10 MB）中嘗試一下。 HTH。