StarCluster - Mailing List Archive

Re: cluster creation best practice

From: Ramon Ramirez-Linan <no email>
Date: Fri, 7 Nov 2014 16:41:39 -0500


You are right, and it is not common to run out of capacity but not
impossible, check the startcluster error bellow.

We use US-EAST region

*> Starting stopped node: node011*
*>>> Starting stopped node: node012*
*>>> Starting stopped node: node013*
*!!! ERROR - InsufficientInstanceCapacity: We currently do not have
sufficient c3.4xlarge capacity in the Availability Zone you requested
(us-east-1b). Our system will be working on provisioning additional
capacity. You can currently get c3.4xlarge capacity by not specifying an
Availability Zone in your request or choosing us-east-1d, us-east-1a,

On Fri, Nov 7, 2014 at 4:22 PM, Rajat Banerjee <>

> Are you requesting a particular type of instance, or in an unusual region
> or AZ?
> I work for AWS and though I have no definitive evidence, I doubt that
> you'd have difficulty launching most types of instances in any of the big
> regions, N. Virginia, Oregon, or Ireland.
> On Fri, Nov 7, 2014 at 2:49 PM, Ramon Ramirez-Linan <>
> wrote:
>> Jennifer,
>> Thanks for your answer although I was aware of all those limits and in
>> fact we requested the limit increased a while ago.
>> My question was more about how to check if AWS has the resources that I
>> need before asking StartCluster to start creating the cluster.
>> If I need a 300 nodes cluster and it fails after 250 due to AWS not
>> having enough availability of an instance type, I will still have to pay
>> for the 250 nodes for at least an hour.
>> After the 250 nodes have been created I have the option of keep trying to
>> add more nodes to complete to 300 but I think to optimize the cost it would
>> be better if I have a way of knowing in advance if AWS is going to have 300
>> EC2 for me.
>> PS: I am an AWS Solutions Architect and my company is an APN and reseller
>> partner.
>> On Thu, Nov 6, 2014 at 12:30 PM, Jennifer Staab <>
>> wrote:
>>> AWS does put limits on resources any one AWS account can use (see here
>>> <> for
>>> details). Specifically see here
>>> <>
>>> for limits regarding EC2s. You can make requests to AWS to increase limit
>>> bounds. Also note there are certain actions AWS users are not allowed take
>>> (see here <> ), if you violate any of those
>>> policies Amazon has right to deny service to you.
>>> And another thing to consider is the time of the year. AWS started
>>> because Amazon wanted to make more use of its resources when they were less
>>> than active. I'm not sure what proportion of AWS resources are still used
>>> by Amazon itself, but I would think with holidays approaching the EC2s
>>> might be less than available as compared to the start of 2015.
>>> Good Luck.
>>> -Jennifer
>>> On 11/6/14 11:41 AM, Ramon Ramirez-Linan wrote:
>>> I need to create a 300 nodes cluster.
>>> 1. if there a way to query AWS in advance to see if they have the 300
>>> EC2 availables? I dont want to find out after 299 nodes that they out of
>>> that EC2 type. I am not sure if StarCluster already check for that.
>>> 2. Would it be better to create the cluster with less nodes and then
>>> add them 20 at a time?
>>> Thanks in advance
>>> Ramon
>>> _______________________________________________
>>> StarCluster mailing listStarCluster_at_mit.edu
>> _______________________________________________
>> StarCluster mailing list
Received on Fri Nov 07 2014 - 16:41:43 EST
This archive was generated by hypermail 2.3.0.


Sort all by: