EC2請求限制超出問題

Question

我正在運行每天啓動（使用JClouds）許多AWS EC2計算機的自動化測試。 問題在於有時機器由於「超出請求限制」錯誤而無法啓動。

從下面的堆棧跟蹤我認爲我有一個失敗5次的操作，5是失敗限制。也許這個動作是jclouds的方法describeInstancesInRegion？爲什麼失敗？

我知道有一個jclouds錯誤，有時會啓動一個空/空名稱的機器。可以在describeInstancesInRegion調用之後破壞它嗎？

堆棧跟蹤：

12/20/13 2:00:26 PM - Cannot retry after server error, **command has exceeded retry limit 5**: [method=public abstract com.google.common.util.concurrent.ListenableFuture org.jclouds.aws.ec2.services.AWSInstanceAsyncClient.**describeInstancesInRegion**(java.lang.String,java.lang.String[])[eu-west-1, [Ljava.lang.String;@5987955b], request=POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1] 
12/20/13 2:00:26 PM - << problem customizing node(eu-west-1/i-38cb4677): : Operation failed. org.jclouds.aws.AWSResponseException: request POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 503, error: AWSError{requestId='8937a5d2-f17f-4a42-af32-ed02125a73a4', requestToken='null', code='**RequestLimitExceeded**', message='Request limit exceeded.', context='{Response=, Errors=}'} 
12/20/13 2:00:26 PM - at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:76) 
12/20/13 2:00:26 PM - at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:69) 
12/20/13 2:00:26 PM - at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:180) 
12/20/13 2:00:26 PM - at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:150) 
12/20/13 2:00:26 PM - at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.invoke(InvokeSyncToAsyncHttpMethod.java:131) 
12/20/13 2:00:26 PM - at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:97) 
12/20/13 2:00:26 PM - at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:58) 
12/20/13 2:00:26 PM - at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117) 
12/20/13 2:00:26 PM - at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:70) 
12/20/13 2:00:26 PM - at $Proxy67.describeInstancesInRegion(Unknown Source) 
12/20/13 2:00:26 PM - at org.jclouds.ec2.compute.strategy.EC2GetNodeMetadataStrategy.getRunningInstanceInRegion(EC2GetNodeMetadataStrategy.java:68) 
12/20/13 2:00:26 PM - at org.jclouds.aws.ec2.compute.strategy.AWSEC2GetNodeMetadataStrategy.getRunningInstanceInRegion(AWSEC2GetNodeMetadataStrategy.java:56) 
12/20/13 2:00:26 PM - at org.jclouds.ec2.compute.strategy.EC2GetNodeMetadataStrategy.getNode(EC2GetNodeMetadataStrategy.java:60) 
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.refreshOrNull(RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.java:60) 
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.refreshOrNull(RefreshNodeAndDoubleCheckOnFailUnlessStatusInvalid.java:40) 
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshAndDoubleCheckOnFailUnlessStatusInvalid.apply(RefreshAndDoubleCheckOnFailUnlessStatusInvalid.java:59) 
12/20/13 2:00:26 PM - at org.jclouds.compute.predicates.internal.RefreshAndDoubleCheckOnFailUnlessStatusInvalid.apply(RefreshAndDoubleCheckOnFailUnlessStatusInvalid.java:42) 
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull$1.apply(ComputeServiceTimeoutsModule.java:148) 
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull$1.apply(ComputeServiceTimeoutsModule.java:143) 
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull.apply(ComputeServiceTimeoutsModule.java:160) 
12/20/13 2:00:26 PM - at org.jclouds.util.Predicates2$RetryablePredicate.apply(Predicates2.java:111) 
12/20/13 2:00:26 PM - at org.jclouds.compute.config.ComputeServiceTimeoutsModule$RetryablePredicateGuardingNull.apply(ComputeServiceTimeoutsModule.java:124) 
12/20/13 2:00:26 PM - at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:65) 
12/20/13 2:00:26 PM - at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:121) 
12/20/13 2:00:26 PM - at org.jclouds.compute.functions.PollNodeRunning.apply(PollNodeRunning.java:44) 
12/20/13 2:00:26 PM - at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
12/20/13 2:00:26 PM - at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:53) 
12/20/13 2:00:26 PM - at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
12/20/13 2:00:26 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
12/20/13 2:00:26 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
12/20/13 2:00:26 PM - at java.lang.Thread.run(Thread.java:662) 
12/20/13 2:00:26 PM - 
12/20/13 2:01:02 PM - Failed to create Cloudify Management VM: Error org.cloudifysource.esc.driver.provisioning.CloudProvisioningException: Failed to start machine. 
12/20/13 2:00:26 PM - JClouds Deployer is shutting down a server that failed to start properly: eu-west-1/i-38cb4677 
12/20/13 2:01:02 PM - Failed to start a management machine: Operation failed. java.util.concurrent.ExecutionException: org.cloudifysource.esc.driver.provisioning.CloudProvisioningException: Failed to start machine 
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver.doStartManagementMachines(BaseProvisioningDriver.java:327) 
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask.get(FutureTask.java:91) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver.startManagementMachines(DefaultProvisioningDriver.java:427) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper.createManagementServers(CloudGridAgentBootstrapper.java:331) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper.getOrCreateManagementServers(CloudGridAgentBootstrapper.java:278) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.commands.BootstrapCloud.doExecute(BootstrapCloud.java:224) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper.bootstrapCloudAndWait(CloudGridAgentBootstrapper.java:203) 
12/20/13 2:01:02 PM - at org.cloudifysource.shell.commands.AbstractGSCommand.execute(AbstractGSCommand.java:103) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.commands.basic.AbstractCommand.execute(AbstractCommand.java:35) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:474) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:400) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:183) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:120) 
12/20/13 2:01:02 PM - at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:89) 
12/20/13 2:01:02 PM - at org.apache.karaf.shell.console.jline.Console.run(Console.java:172) 
12/20/13 2:01:02 PM - at org.apache.karaf.shell.console.Main.run(Main.java:191) 
12/20/13 2:01:02 PM - at org.apache.karaf.shell.console.Main.run(Main.java:89) 
12/20/13 2:01:02 PM - at org.cloudifysource.shell.GigaShellMain.main(GigaShellMain.java:126) 
12/20/13 2:01:02 PM - Caused by: org.cloudifysource.esc.driver.provisioning.CloudProvisioningException: Failed to start machine 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.jclouds.JCloudsDeployer.createServer(JCloudsDeployer.java:178) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver.createServer(DefaultProvisioningDriver.java:260) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.jclouds.DefaultProvisioningDriver.createServer(DefaultProvisioningDriver.java:236) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver$1.call(BaseProvisioningDriver.java:315) 
12/20/13 2:01:02 PM - at org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver$1.call(BaseProvisioningDriver.java:310) 
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
12/20/13 2:01:02 PM - at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
12/20/13 2:01:02 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
12/20/13 2:01:02 PM - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
12/20/13 2:01:02 PM - at java.lang.Thread.run(Thread.java:662) 
12/20/13 2:01:02 PM - Caused by: org.jclouds.compute.RunNodesException: error running 1 node group(ec2-user-master-ec2gitservicestest-cloudify-manager1) location(eu-west-1) image(ami-c37474b7) size(m1.small) options({groupNames=[default], keyPair=ec2-sgtest-eu, userDataCksum=2f4a740b}) 

12/20/13 2:01:02 PM - Execution failures: 
12/20/13 2:01:02 PM - 
12/20/13 2:01:02 PM - 0 error[s] 
12/20/13 2:01:02 PM - Node failures: 
12/20/13 2:01:02 PM - 
12/20/13 2:01:02 PM - 1) AWSResponseException on node eu-west-1/i-38cb4677: 
12/20/13 2:01:02 PM - org.jclouds.aws.AWSResponseException: request POST https://ec2.eu-west-1.amazonaws.com/ HTTP/1.1 failed with code 503, error: AWSError{requestId='8937a5d2-f17f-4a42-af32-ed02125a73a4', requestToken='null', code='RequestLimitExceeded', message='Request limit exceeded.', context='{Response=, Errors=}'} 

jclouds version is: 1.6.2-incubating 

Answer 1

你需要告訴jclouds不提出要求卻常常是輪詢例如狀態更新。你可以用下面的覆蓋來做到這一點。

Properties overrides = new Properties(); 
overrides.setProperty(POLL_INITIAL_PERIOD, POLL_PERIOD_TWENTY_SECONDS); 
overrides.setProperty(POLL_MAX_PERIOD, POLL_PERIOD_TWENTY_SECONDS); 

ComputeServiceContext context = ContextBuilder.newBuilder(PROVIDER) 
     .credentials(username, apiKey) 
     .overrides(overrides) 
     .buildView(ComputeServiceContext.class); 
computeService = context.getComputeService(); 

查看完整的示例，看看CloudServersPublish。儘管這是Rackspace Cloud的一個例子，但代碼幾乎相同。