谷歌雲存儲python客戶端的批量請求

Question

我找不到任何關於如何使用python谷歌雲存儲的批處理功能的例子。我看到它存在here。

我很喜歡一個具體的例子。比方說，我想要刪除一組具有給定前綴的blob。我開始越來越斑點的名單如下

from google.cloud import storage 

storage_client = storage.Client() 
bucket = storage_client.get_bucket('my_bucket_name') 
blobs_to_delete = bucket.list_blobs(prefix="my/prefix/here") 

# how do I delete the blobs in blobs_to_delete in a single batch? 

# bonus: if I have more than 100 blobs to delete, handle the limitation 
#  that a batch can only handle 100 operations 

Answer 1

TL; DR - 只需發送的所有batch() context manager範圍內的請求（可在google-cloud-python庫）

試試這個例子：

from google.cloud import storage 

storage_client = storage.Client() 
bucket = storage_client.get_bucket('my_bucket_name') 
# Accumulate the iterated results in a list prior to issuing 
# batch within the context manager 
blobs_to_delete = [blob for blob in bucket.list_blobs(prefix="my/prefix/here")] 

# Use the batch context manager to delete all the blobs  
with storage_client.batch(): 
    for blob in blobs: 
     blob.delete() 

如果您直接使用REST API，則只需要擔心每批次的100個項目。 batch() context manager會自動處理此限制，並在需要時發出多個批處理請求。