RejectedExecutionException使用AWS TransferManager時

我有以下方法做多上傳到amazonS3：RejectedExecutionException使用AWS TransferManager時

public static void uploadToS3UserPath(String filePath, String bucket, String userPath) { 
    ClientConfiguration config = new ClientConfiguration(); 
    config.setProxyHost("host"); 
    config.setProxyPort(3128); 
    config.setProtocol(Protocol.HTTP); 

    TransferManager tm = new TransferManager(new AmazonS3Client(credentials, config)); 
    TransferManagerConfiguration conf = new TransferManagerConfiguration(); 
    conf.setMinimumUploadPartSize(50 * 1024 * 1024); //use 50 megabytes parts; 
    tm.setConfiguration(conf); 

    PutObjectRequest req = new PutObjectRequest(bucket, userPath, new File(filePath)); 
    req.setCannedAcl(CannedAccessControlList.PublicRead); 
    req.setStorageClass(StorageClass.ReducedRedundancy); //we do it to decrease storage costs 
    Upload up = tm.upload(req); 

    try { 
     up.waitForCompletion(); 
     tm.shutdownNow(); 
     System.out.println("Upload completed successfully"); 
    } catch (AmazonClientException ex) { 
     Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, ex.getLocalizedMessage(), ex); 
     tm.shutdownNow(); 
    } catch (InterruptedException ex) { 
     Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, ex.getLocalizedMessage(), ex); 
    } 
}

事情有時我得到：

com.amazonaws.AmazonClientException: Unable to complete transfer: null 
      at 
    com.amazonaws.services.s3.transfer.Transfer.unwrapExecutionException(Transfer.java:226) 
      at 
    com.amazonaws.services.s3.transfer.Transfer.rethrowExecutionException(Transfer.java:210) 
      at 
    com.amazonaws.services.s3.transfer.Transfer.waitForCompletion(Transfer.java:116) 
      at 
    xx.xxx.xxx.xxx.agent.Utilities.uploadToS3UserPath(Utilities.java:373) 
      at 
    xx.xxx.xxx.xxx.agent.AbstractAgent.uploadOutputToS3(AbstractAgent.java:312) 
      at 
    uk.org.infectogenomics.agent.AbstractAgent.uploadOutputToS3(AbstractAgent.java:305) 
      at xx.xxx.xxx.xxx.hostEl.HostEl.run(HostEl.java:598) 
      at xx.xxx.xxx.xxx.agent.Agent.main(Agent.java:98) 
    Caused by: java.util.concurrent.RejectedExecutionException 
      at 
    java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1759) 
      at 
    java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767) 
      at 
    java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:216) 
      at 
    java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:378) 
      at 
    com.amazonaws.services.s3.transfer.internal.UploadMonitor.reschedule(UploadMonitor.java:210) 
      at 
    com.amazonaws.services.s3.transfer.internal.UploadMonitor.upload(UploadMonitor.java:197) 
      at 
    com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:148) 
      at 
    com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:49) 
      at 
    java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
      at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
      at 
    java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) 
      at 
    java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) 
      at java.lang.Thread.run(Thread.java:619)

的事情是我不知道爲什麼這只是爲了隨機上傳而顯示。顯然，原因是RejectedExecutionException？難道是因爲我提前關閉了TransferManager？

來源

2011-10-07 LordDoskias

的比如你有沒有弄清楚發生了什麼事情？ –

使用多線程實現最終證明是很難得到正確的，無論他們有非常漂亮的API的事實，所以我只是恢復到使用單線程版本 - 與uploadrequest。因爲我使用的是ja.net，所以我們與雲有很好的連接性，因此將幾個gig文件傳輸到亞馬遜並不是一件好事 – LordDoskias

難道是因爲我提前關閉了TransferManager嗎？

這確實是該行爲觸發的，但根本原因就在AWS SDK for Java是一個錯誤，請參閱埃裏克·米拉斯類似的問題Amazon AWS Java SDK TransferManager issue：

TransferManager包含UploadMonitor和UploadMonitor有靜態ScheduledExecutorService的稱爲timedThreadPoool。 TransferManager.shutdownNow（）調用 UploadMonitor.shutdownNow（），它調用 timedThreadPoool.shutdownNow（）。在下次調用 UploadMonitor.reschedule（）時，拋出的RejectedExecutionException爲。

His own answer引用AWS團隊應對TransferManager Thread Issue，這證實了這個bug，以及相應的修正，已經實現的Release 1.3.2）：

的Amazon S3 TransferManager class進行了更新，以解決兩個錯誤：

從InputStream上傳時可能會發生數據損壞，而在上載的某些部分期間遇到IO錯誤時會出現。乾淨關閉計時器線程時shutdownNow時（）調用TransferManager

來源

2013-10-23 18:31:37

RejectedExecutionException使用AWS TransferManager時

回答

相關問題