Re: [Starcluster] Spot instances

From: Justin Riley <no email>
Date: Tue, 30 Mar 2010 22:50:00 -0400

Hi Joonas,

Excellent, glad you got it working. Thanks for providing your steps. I'll add
a section to the new docs about using spot instances.

Thanks for your help and for testing the github version,

~Justin

On Tuesday 30 March 2010 3:56:16 pm Joonas Paalasmaa wrote:
> Hi,
>
> Thanks for the help. I did get a spot instance cluster working with
> the following commands and latest git version.
>
> - Joonas
>
> ----
>
> ec2-add-group "_at_sc-cluster3" -d "StarCluster cluster3"
> ec2-authorize "_at_sc-cluster3" -P tcp -p 22 -s 0.0.0.0/0
> ec2-authorize "_at_sc-cluster3" -P tcp -p 1-65535 -o "@sc-cluster3" -u
> MY_USERID ec2-authorize "_at_sc-cluster3" -P udp -p 1-65535 -o "@sc-cluster3"
> -u MY_USERID ec2-authorize "_at_sc-cluster3" -P icmp -t -1:-1 -o
> "_at_sc-cluster3" -u MY_USERID ec2-request-spot-instances ami-8f9e71e6
> --group "_at_sc-cluster3" -m -z us-east-1a -p 1.0 -t m1.small -n 1 -r
> one-time -k pstam-keypair
> --group "_at_sc-masters"
> ec2-request-spot-instances ami-8f9e71e6 --group "_at_sc-cluster3" -m -z
> us-east-1a -p 1.0 -t m1.small -n 2 -r one-time -k pstam-keypair
>
> # Wait until the instances are "pending"
>
> starcluster start -x -s 3 small cluster3
>
> On Tue, Mar 30, 2010 at 4:41 PM, Justin Riley <jtriley_at_mit.edu> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Hi Joonas,
> >
> > It appears that your cluster template high-cpu-xl has CLUSTER_SIZE=1
> > given that it extends the "small" cluster template in your config.
> >
> > I believe your issue has to do with the cluster size in the template not
> > being the same as the number of instances launched in the _at_sc-cluster1
> > security group. One way to fix this would be to put CLUSTER_SIZE=4 in
> > your "high-cpu-xl" cluster template.
> >
> > A quick fix would be to run:
> >
> > $ starcluster start -x -s 4 high-cpu-xl cluster1
> >
> > Let me know if that works for you or not.
> >
> > Thanks,
> >
> > ~Justin
> >
> > On 03/30/2010 09:00 AM, Joonas Paalasmaa wrote:
> >> Hi,
> >>
> >> I tried to set up a cluster as you suggested.
> >>
> >> First, I started four instances with the commands below.
> >>
> >> ec2-add-group "_at_sc-cluster1" -d "StarCluster cluster1"
> >> ec2-authorize "_at_sc-cluster1" -P tcp -p 22 -s 0.0.0.0/0
> >> ec2-request-spot-instances ami-a19e71c8 --group "_at_sc-masters" --group
> >> "_at_sc-cluster1" -m -z us-east-1a -p 1.0 -t c1.xlarge -n 1 -r one-time
> >> ec2-request-spot-instances ami-a19e71c8 --group "_at_sc-cluster1" -m -z
> >> us-east-1a -p 1.0 -t c1.xlarge -n 3 -r one-time
> >> starcluster start -x high-cpu-xl cluster1
> >>
> >> Once the instances were running, I ran the command "starcluster start
> >> -x high-cpu-xl cluster1". However, it did not go beyond the text
> >> "Waiting for cluster to start...".
> >>
> >> What could I be doing wrong?
> >>
> >> - Joonas
> >>
> >>
> >> Here's my config for the cluster:
> >>
> >>
> >> [cluster small]
> >> KEYNAME = pstam-keypair
> >>
> >> CLUSTER_SIZE = 1
> >>
> >> CLUSTER_USER = jpaalasm
> >> CLUSTER_SHELL = bash
> >>
> >> NODE_IMAGE_ID = ami-8f9e71e6
> >> NODE_INSTANCE_TYPE = m1.small
> >> AVAILABILITY_ZONE = us-east-1a
> >> VOLUMES = shhs
> >>
> >>
> >> [cluster high-cpu-xl]
> >> EXTENDS = small
> >> NODE_IMAGE_ID = ami-a19e71c8
> >> NODE_INSTANCE_TYPE = c1.xlarge
>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
Received on Tue Mar 30 2010 - 22:50:02 EDT