2015-10-14 65 views
4

我需要在AWS lambda函數中使用boto3 python將.zip文件從S3轉換爲.gzip文件。有關如何做到這一點的任何建議?使用AWS lambda函數將S3文件從zip文件轉換爲gzip使用boto3 python

這是我到目前爲止有:

import json 
import boto3 
import zipfile 
import gzip 

s3 = boto3.resource('s3') 

def lambda_handler(event, context): 

    bucket = event['Records'][0]['s3']['bucket']['name'] 
    key = event['Records'][0]['s3']['object']['key'] 

    try: 
     s3Obj = s3.Object(bucket_name=bucket, key=key) 
     response = s3Obj.get() 
     data = response['Body'].read() 
     zipToGzip = gzip.open(data, 'wb') 
     zipToGzip.write(s3.upload_file(bucket, (s3 + '.gz'))) 
     zipToGzip.close() 
    except Exception as e: 
     print(e) 
     print('Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket)) 
     raise e 
+0

更多詳細信息和您當前的代碼將有所幫助。你想重新上傳文件到S3 gziped,或者只是在本地gzip上做些什麼?爲什麼它必須是lambda函數?你的意思是Python lambda,還是AWS lambda? – Lee

+0

我的意思是使用Python的AWS Lambda函數,因爲它現在支持。我在S3上有一個.zip格式的文件,我需要將其更改爲.gzip格式。 – Scotty

+0

太好了,謝謝你的澄清。當前的代碼會發生什麼?它提出了一個例外,或不做你想要的......? – Lee

回答

6

OK,想通弄明白了。感謝您的意見。

import json 
import boto3 
import zipfile 
import gzip 

print('Loading function') 

s3 = boto3.resource('s3') 
s3_client = boto3.client('s3') 

def lambda_handler(event, context): 

    # Get the object from the event and show its content type 
    bucket = event['Records'][0]['s3']['bucket']['name'] 
    key = event['Records'][0]['s3']['object']['key'] 

    try: 
     s3_client.download_file(bucket, key, '/tmp/file.zip') 
     zfile = zipfile.ZipFile('/tmp/file.zip') 
     namelist = zfile.namelist() 

     if len(namelist) >1: 
      pass 
      #alertme() 

     for filename in namelist: 
      data = zfile.read(filename) 
      f = open('/tmp/' + str(filename), 'wb') 
      f.write(data) 
      f.close() 

     zipToGzip = gzip.open('/tmp/data.gz', 'wb') 
     zipToGzip.write(data) 
     zipToGzip.close() 
     s3_client.upload_file('/tmp/data.gz', bucket, key + '.gz') 
     s3_client.delete_object(Bucket=bucket, Key=key) 
    except Exception as e: 
     print(e) 
     print('Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket)) 
     raise e 
+0

問題是什麼?你的代碼有很多變化。但是將對象保存到一個臨時文件中修復它? – Lee

+0

@Lee,我很確定問題是Boto3對象沒有被正確讀爲二進制。因此Scotty必須從S3存儲桶下載文件,而不是使用'response ['Body'] .read()'獲取 - 這應該是二進制文件的內容。我有同樣的問題 - 試圖打印'response ['Body']。read()'只是給了我字符串「PK」。不知道這是什麼意思。不得不從桶裏拉出來很煩人。 – unclemeat

相關問題