的Python Boto3：錯誤時，試圖從AWS S3

我試圖下載從S3存儲是非常大的所有文件下載文件。我連接到S3這樣的：

client = boto3.client('s3', 
aws_access_key_id=tempCredentials.credentials.access_key, 
aws_secret_access_key = tempCredentials.credentials.secret_key,         
aws_session_token=tempCredentials.credentials.session_token)

從這個，我做的：

# This is going to go through and fill in the dictionary with keys 
from the buckets as specified above 
paginator = client.get_paginator("list_objects") 
page_iterator = paginator.paginate(Bucket=bucket["Name"]) 
l = 0 
# We are going to have an list that will hold all the keys 
key_list = [] 
for i in page_iterator: 
    c = i["Contents"] 
    for j in c: 
      key_list.append(j["Key"]) 
    for j in key_list: 
     download(bucket["Name"], j, "/Users/ahussain/Desktop/S3_Scrubber/" + file_name_helper(j), client)

在哪裏，我的下載功能爲：

def download (bucket_name, key, path, client): 
    key_name = key 
    print("Dowloading %s..." % str(key)) 
    client.download_file(bucket_name, key, path) 
    print("Download of %s complete!" % str(key)) 
    return key_name

什麼情況是，我經歷該桶成功並下載了大量的密鑰，但過了一段時間程序停止下載密鑰並給我出現此錯誤：

botocore.exceptions.ClientError: An error occurred (400) when calling the HeadObject operation: Bad Request

我的猜測是我的會話因我使用MFA訪問S3而過期，但我不確定。有沒有人曾經遇到過這個錯誤？

來源

2017-08-02 user3047661

在構建客戶端時，您可以嘗試指定您的存儲桶所在的區域嗎？ 'client = boto3.client（'s3'，region_name ='us-west-1'，....' – helloV

臨時憑證僅在一小時內有效。從IAM documentation：

The duration, which specifies how long the temporary security credentials are valid. The minimum is 15 minutes (900 seconds) and the maximum (and the default) is 1 hour (3600 seconds). You need to pass this value only if you want the temporary credentials to expire before 1 hour.

根據公開的bug（https://github.com/boto/boto3/issues/443）boto3不支持臨時憑證的清爽長時間運行的操作。

因此，如果您的腳本在〜1小時後顯示錯誤，可能是原因。

來源

2017-08-02 19:22:19 Dunedan

的Python Boto3：錯誤時，試圖從AWS S3

回答

相關問題