>
> We do have support for even more fine grained control of node instance
> types (thanks to Dan Yamins) if it's needed,
The syntax is pretty simple (correct me where I go wrong, Justin). It is
basically governed y the following rules:
1) To specify N instance-types "type1', ... "typeN", with specific AMIs
and number of nodes for each type, you use a comma-separated list of
elements, with ":"-separation between declarations for instance type, AMI
ID, and number of nodes, e.g.:
NODE_INSTANCE_TYPE = type1 : ami1 : num1 , type2 : ami2 : num 2, ... ,
typeN : amiN : numN
2) The AMI specification is optional, e.g.
NODE_INSTANCE_TYPE = type1 : num1 , type2 : num 2, ... , typeN : numN
is a valid specification, with the effect that all instances will use the
ami specified by NODE_IMAGE_ID.
3) The num specification is optional as well, e.g.
NODE_INSTANCE_TYPE = type1, type2, ... , typeN
is a valid specification, with the effect of bringing up 1 node each for
type1, type2, ...typeN-1. If numN is unspecified, it is automatically set
to
numN = CLUSTER_SIZE - num1 - num2 ... - numN-1
4) The numbers num1, num2, ... numN, must satisfy;
CLUSTER_SIZE = num1 + num2 + ... + numN
If you specify too many or too few nodes, an error will be thrown.
Examples:
######### this sets up a 4-node cluster with 1 m1.large, 1 c1.xlarge, and
3 m1.xlarge
CLUSTER_SIZE = 5
NODE_INSTANCE_TYPE = m1.large,c1.xlarge,m1.xlarge
#########
######### this sets up a 10-node cluster with 4 m1.large, 4 c1.xlarge, and
2 m1.xlarge
CLUSTER_SIZE = 10
NODE_INSTANCE_TYPE = m1.large : 4 , c1.xlarge : 4 ,m1.xlarge
#########
######### this sets up a 10-node cluster with 4 m1.large using ami
ami-7aca2213,
######### and 6 c1.xlarge using the AMI specified by NODE_IMAGE ID
CLUSTER_SIZE = 10
NODE_INSTANCE_TYPE = m1.large : ami-7aca2213: 4 , c1.xlarge
#########
######### this generates an error
CLUSTER_SIZE = 5
NODE_INSTANCE_TYPE = m1.large : 4 , c1.xlarge : 4
#########
If yo have any questions, feel free to ask
Dan
> however, I haven't yet documented this functionality. Let me know if the
> above doesn't meet your needs and I'll fill you in.
>
> ~Justin
> ________________________________________
> From: starcluster-bounces_at_MIT.EDU [starcluster-bounces_at_MIT.EDU] On Behalf
> Of Hesselberth, Jay [JAY.HESSELBERTH_at_ucdenver.edu]
> Sent: Monday, August 09, 2010 3:30 PM
> To: starcluster_at_mit.edu
> Subject: [Starcluster] Cluster with mixed NODE_INSTANCE_TYPE's
>
> I have an application where I'm running 1000's of jobs with small memory
> requirements (e.g. ~500 Mb / job), but there is a periodic job that needs
> substantially more RAM (~8 Gb). I've been running clusters with c1.xlarge
> instances to maximize throughput of the small jobs, but these don't have the
> necessary RAM.
>
> I was thinking it would be nice to specify clusters with mixed
> NODE_INSTANCE_TYPES; e.g. configuring a cluster with 4 c1.xlarge instances
> for throughput and 1 m2.xlarge instance to handle large RAM jobs. I don't
> see how this is currently possible (I'm using the experimental starcluster
> branch). Is a feature that is planned for the future?
>
> Thanks,
> Jay Hesselberth
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
Received on Mon Aug 09 2010 - 17:16:23 EDT