如何使用PHP和aws-sdk v3將大型檔案上傳到Amazon Glacier？

Question

這是我第一次與亞馬遜的任何工作。我正嘗試使用PHP SDK V3將多個文件上傳到Amazon Glacier。這些文件將需要由亞馬遜合併爲一個。

這些文件存儲在cPanel的主目錄中，並且必須通過cron作業上傳到Amazon Glacier。

我知道我必須使用上傳多部分方法，但我不確定它需要哪些其他功能才能使其工作。我也不確定我計算和通過變量的方式是否正確。

這是我到目前爲止的代碼：

<?php 
require 'aws-autoloader.php'; 

use Aws\Glacier\GlacierClient; 
use Aws\Glacier\TreeHash; 

//############################################ 
//DEFAULT VARIABLES 
//############################################ 
$key = 'XXXXXXXXXXXXXXXXXXXX'; 
$secret = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'; 
$accountId = '123456789123'; 
$vaultName = 'VaultName'; 
$partSize = '4194304'; 
$fileLocation = 'path/to/files/'; 

//############################################ 
//DECLARE THE AMAZON CLIENT 
//############################################ 
$client = new GlacierClient([ 
    'region' => 'us-west-2', 
    'version' => '2012-06-01', 
    'credentials' => array(
     'key' => $key, 
     'secret' => $secret, 
) 
]); 

//############################################ 
//GET THE UPLOAD ID 
//############################################ 
$result = $client->initiateMultipartUpload([ 
    'partSize' => $partSize, 
    'vaultName' => $vaultName 
]); 
$uploadId = $result['uploadId']; 

//############################################ 
//GET ALL FILES INTO AN ARRAY 
//############################################ 
$files = scandir($fileLocation); 
unset($files[0]); 
unset($files[1]); 
sort($files); 

//############################################ 
//GET SHA256 TREE HASH (CHECKSUM) 
//############################################ 
$th = new TreeHash(); 
//GET TOTAL FILE SIZE 
foreach($files as $part){ 
    $filesize = filesize($fileLocation.$part); 
    $total = $filesize; 
    $th = $th->update(file_get_contents($fileLocation.$part)); 
} 
$totalchecksum = $th->complete(); 

//############################################ 
//UPLOAD FILES 
//############################################ 
foreach ($files as $key => $part) { 
    //HASH CONTENT 
    $filesize = filesize($fileLocation.$part); 
    $rangeSize = $filesize-1; 
    $range = 'bytes 0-'.$rangeSize.'/*'; 
    $sourcefile = $fileLocation.$part; 

    $result = $client->uploadMultipartPart([ 
     'accountId' => $accountId, 
     'checksum' => '', 
     'range' => $range, 
     'sourceFile' => $sourcefile, 
     'uploadId' => $uploadId, 
     'vaultName' => $vaultName 
    ]); 
} 

//############################################ 
//COMPLETE MULTIPART UPLOAD 
//############################################ 
$result = $client->completeMultipartUpload([ 
    'accountId' => $accountId, 
    'archiveSize' => $total, 
    'checksum' => $totalchecksum, 
    'uploadId' => $uploadId, 
    'vaultName' => $vaultName, 
]); 
?> 

看來，一個新的冰河客戶端的申報工作和我收到UPLOADID，但其餘的，如果我我不是100％我做對了。 Amazon Glacier Vault文件需要上傳併合並後保留爲空，我不確定文件是否僅顯示completeMultipartUpload已成功執行。

Fatal error: Uncaught exception 'Aws\Glacier\Exception\GlacierException' with message 'Error executing "CompleteMultipartUpload" on " https://glacier.us-west-2.amazonaws.com/XXXXXXXXXXXX/vaults/XXXXXXXXXX/multipart-uploads/cTI0Yfk6xBYIQ0V-rhq6AcdHqd3iivRJfyYzK6-NV1yn9GQvJyYCoSrXrrrx4kfyGm6m9PUEAq4M0x6duXm5MD8abn-M "; AWS HTTP error: Client error: 403 InvalidSignatureException (client): The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details. The Canonical String for this request should have been 'POST /XXXXXXXXXXX/vaults/XXXXXXXXX/multipart-uploads/cTI0Yfk6xBYIQ0V-rhq6AcdHqd3iivRJfyYzK6-NV1yn9GQvJyYCoSrXrrrx4kfyGm6m9PUEAq4M0x6duXm5MD8abn-M host:glacier.us-west-2.amazonaws.com x-amz-archive-size:1501297 x-amz-date:20151016T081455Z x-amz-glacier-version:2012-06-01 x-amz-sha256-tree-hash:?[ qiuã°²åÁ¹ý+¤Üª¤ [;K×T host;x-amz-archive-size;x-amz-date;x-amz-glacier-version;x-am in /home/XXXXXXXXXXXX/public_html/XXXXXXXXXXX/Aws/WrappedHttpHandler.php on line 152

是否有可能一個簡單的方法來做到這一點：

運行的代碼時，我還收到以下錯誤？如果有幫助，我也有完整的SSH訪問權限。

Answer 1

我想你誤會了uploadMultipartPart。 uploadMultipartPart表示您上傳1個大文件，分爲多個部分。 ，然後執行completeMultipartUpload以標記您已完成上傳一個文件。

從您的代碼，它看起來像你正在上傳多個文件。

有可能是你實際上並不需要使用uploadMultipartPart

也許你可以使用常規的「uploadArchive」？

裁判：

https://blogs.aws.amazon.com/php/post/Tx7PFHT4OJRJ42/Uploading-Archives-to-Amazon-Glacier-from-PHP

Answer 2

我在PHP SDK V3管理本（第3版），我一直在尋找我的研究這個問題，所以我想我會發布我的解決辦法了。使用需要您自擔風險，並且很少有錯誤檢查或處理。

<?php 
require 'vendor/autoload.php'; 

use Aws\Glacier\GlacierClient; 
use Aws\Glacier\TreeHash; 


// Create the glacier client to connect with 
$glacier = new GlacierClient(array(
     'profile' => 'default', 
     'region' => 'us-east-1', 
     'version' => '2012-06-01' 
    )); 

$fileName = '17mb_test_file';   // this is the file to upload 
$chunkSize = 1024 * 1024 * pow(2,2); // 1 MB times a power of 2 
$fileSize = filesize($fileName);  // we will need the file size (in bytes) 

// initiate the multipart upload 
// it is dangerous to send the filename without escaping it first 
$result = $glacier->initiateMultipartUpload(array(
     'archiveDescription' => 'A multipart-upload for file: '.$fileName, 
     'partSize' => $chunkSize, 
     'vaultName' => 'MyVault' 
    )); 

// we need the upload ID when uploading the parts 
$uploadId = $result['uploadId']; 

// we need to generate the SHA256 tree hash 
// open the file so we can get a hash from its contents 
$fp = fopen($fileName, 'r'); 
// This class can generate the hash 
$th = new TreeHash(); 
// feed in all of the data 
$th->update(fread($fp, $fileSize)); 
// generate the hash (this comes out as binary data)... 
$hash = $th->complete(); 
// but the API needs hex (thanks). PHP to the rescue! 
$hash = bin2hex($hash); 

// reset the file position indicator 
fseek($fp, 0); 

// the part counter 
$partNumber = 0; 

print("Uploading: '".$fileName 
    ."' (".$fileSize." bytes) in " 
    .(ceil($fileSize/$chunkSize))." parts...\n"); 
while ($partNumber * $chunkSize < ($fileSize + 1)) 
{ 
    // while we haven't written everything out yet 
    // figure out the offset for the first and last byte of this chunk 
    $firstByte = $partNumber * $chunkSize; 
    // the last byte for this piece is either the last byte in this chunk, or 
    // the end of the file, whichever is less 
    // (watch for those Obi-Wan errors) 
    $lastByte = min((($partNumber + 1) * $chunkSize) - 1, $fileSize - 1); 

    // upload the next piece 
    $result = $glacier->uploadMultipartPart(array(
     'body' => fread($fp, $chunkSize), // read the next chunk 
     'uploadId' => $uploadId,   // the multipart upload this is for 
     'vaultName' => 'MyVault', 
     'range' => 'bytes '.$firstByte.'-'.$lastByte.'/*' // weird string 
     )); 

    // this is where one would check the results for error. 
    // This is left as an exercise for the reader ;) 

    // onto the next piece 
    $partNumber++; 
    print("\tpart ".$partNumber." uploaded...\n"); 
} 
print("...done\n"); 

// and now we can close off this upload 
$result = $glacier->completeMultipartUpload(array(
    'archiveSize' => $fileSize,   // the total file size 
    'uploadId' => $uploadId,   // the upload id 
    'vaultName' => 'MyVault', 
    'checksum' => $hash     // here is where we need the tree hash 
)); 

// this is where one would check the results for error. 
// This is left as an exercise for the reader ;) 


// get the archive id. 
// You will need this to refer to this upload in the future. 
$archiveId = $result->get('archiveId'); 

print("The archive Id is: ".$archiveId."\n"); 


?>