StarCluster - Mailing List Archive

Re: [Starcluster] Cluster with mixed NODE_INSTANCE_TYPE's

From: Justin Riley <no email>
Date: Mon, 9 Aug 2010 16:39:56 -0400

Hi Jay,

Since you're just wanting a single m2.xlarge instance and 4 c1.xlarge instances you could do the following:

[cluster mycluster]
cluster_size = 5
master_instance_type = m2.xlarge
node_instance_type = c1.xlarge

This template will use the m2.xlarge type to launch the master, and the c1.xlarge type for the 4 worker nodes.

We do have support for even more fine grained control of node instance types (thanks to Dan Yamins) if it's needed, however, I haven't yet documented this functionality. Let me know if the above doesn't meet your needs and I'll fill you in.

From: starcluster-bounces_at_MIT.EDU [starcluster-bounces_at_MIT.EDU] On Behalf Of Hesselberth, Jay []
Sent: Monday, August 09, 2010 3:30 PM
Subject: [Starcluster] Cluster with mixed NODE_INSTANCE_TYPE's

I have an application where I'm running 1000's of jobs with small memory requirements (e.g. ~500 Mb / job), but there is a periodic job that needs substantially more RAM (~8 Gb). I've been running clusters with c1.xlarge instances to maximize throughput of the small jobs, but these don't have the necessary RAM.

I was thinking it would be nice to specify clusters with mixed NODE_INSTANCE_TYPES; e.g. configuring a cluster with 4 c1.xlarge instances for throughput and 1 m2.xlarge instance to handle large RAM jobs. I don't see how this is currently possible (I'm using the experimental starcluster branch). Is a feature that is planned for the future?

Jay Hesselberth
Starcluster mailing list
Received on Mon Aug 09 2010 - 16:39:59 EDT
This archive was generated by hypermail 2.3.0.


Sort all by: