-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi All,
I've added *experimental* spot instance support to the starcluster
github code.
To try using it without modifying your configs:
$ starcluster start --bid 0.10 smallcluster mycluster
This will start a cluster named 'mycluster' based on the smallcluster
config template using spot instances with a max bid of $0.10
If you'd prefer to always use spot instances, simply add:
SPOT_BID=0.10
To your cluster template in the config file.
Also, you might want to look at the new spothistory command:
$ starcluster spothistory m1.small
The above command shows you the current, max, and average spot price by
default the last 30 days for the m1.small instance type. If you want
more/less time than that look at the --days-ago option.
There is also an option to plot the history using matplotlib:
$ starcluster spothistory -d 10 m1.small -p
The above will use matplotlib to show you a plot of the spot price
history over the last 10 days.
Please let me know if spot instances and the spothistory command work
for you and any issues you run into.
Thanks,
~Justin
On 03/30/2010 10:50 PM, Justin Riley wrote:
> Hi Joonas,
>
> Excellent, glad you got it working. Thanks for providing your steps. I'll add
> a section to the new docs about using spot instances.
>
> Thanks for your help and for testing the github version,
>
> ~Justin
>
> On Tuesday 30 March 2010 3:56:16 pm Joonas Paalasmaa wrote:
>> Hi,
>>
>> Thanks for the help. I did get a spot instance cluster working with
>> the following commands and latest git version.
>>
>> - Joonas
>>
>> ----
>>
>> ec2-add-group "_at_sc-cluster3" -d "StarCluster cluster3"
>> ec2-authorize "_at_sc-cluster3" -P tcp -p 22 -s 0.0.0.0/0
>> ec2-authorize "_at_sc-cluster3" -P tcp -p 1-65535 -o "@sc-cluster3" -u
>> MY_USERID ec2-authorize "_at_sc-cluster3" -P udp -p 1-65535 -o "@sc-cluster3"
>> -u MY_USERID ec2-authorize "_at_sc-cluster3" -P icmp -t -1:-1 -o
>> "_at_sc-cluster3" -u MY_USERID ec2-request-spot-instances ami-8f9e71e6
>> --group "_at_sc-cluster3" -m -z us-east-1a -p 1.0 -t m1.small -n 1 -r
>> one-time -k pstam-keypair
>> --group "_at_sc-masters"
>> ec2-request-spot-instances ami-8f9e71e6 --group "_at_sc-cluster3" -m -z
>> us-east-1a -p 1.0 -t m1.small -n 2 -r one-time -k pstam-keypair
>>
>> # Wait until the instances are "pending"
>>
>> starcluster start -x -s 3 small cluster3
>>
>> On Tue, Mar 30, 2010 at 4:41 PM, Justin Riley <jtriley_at_mit.edu> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> Hi Joonas,
>>>
>>> It appears that your cluster template high-cpu-xl has CLUSTER_SIZE=1
>>> given that it extends the "small" cluster template in your config.
>>>
>>> I believe your issue has to do with the cluster size in the template not
>>> being the same as the number of instances launched in the _at_sc-cluster1
>>> security group. One way to fix this would be to put CLUSTER_SIZE=4 in
>>> your "high-cpu-xl" cluster template.
>>>
>>> A quick fix would be to run:
>>>
>>> $ starcluster start -x -s 4 high-cpu-xl cluster1
>>>
>>> Let me know if that works for you or not.
>>>
>>> Thanks,
>>>
>>> ~Justin
>>>
>>> On 03/30/2010 09:00 AM, Joonas Paalasmaa wrote:
>>>> Hi,
>>>>
>>>> I tried to set up a cluster as you suggested.
>>>>
>>>> First, I started four instances with the commands below.
>>>>
>>>> ec2-add-group "_at_sc-cluster1" -d "StarCluster cluster1"
>>>> ec2-authorize "_at_sc-cluster1" -P tcp -p 22 -s 0.0.0.0/0
>>>> ec2-request-spot-instances ami-a19e71c8 --group "_at_sc-masters" --group
>>>> "_at_sc-cluster1" -m -z us-east-1a -p 1.0 -t c1.xlarge -n 1 -r one-time
>>>> ec2-request-spot-instances ami-a19e71c8 --group "_at_sc-cluster1" -m -z
>>>> us-east-1a -p 1.0 -t c1.xlarge -n 3 -r one-time
>>>> starcluster start -x high-cpu-xl cluster1
>>>>
>>>> Once the instances were running, I ran the command "starcluster start
>>>> -x high-cpu-xl cluster1". However, it did not go beyond the text
>>>> "Waiting for cluster to start...".
>>>>
>>>> What could I be doing wrong?
>>>>
>>>> - Joonas
>>>>
>>>>
>>>> Here's my config for the cluster:
>>>>
>>>>
>>>> [cluster small]
>>>> KEYNAME = pstam-keypair
>>>>
>>>> CLUSTER_SIZE = 1
>>>>
>>>> CLUSTER_USER = jpaalasm
>>>> CLUSTER_SHELL = bash
>>>>
>>>> NODE_IMAGE_ID = ami-8f9e71e6
>>>> NODE_INSTANCE_TYPE = m1.small
>>>> AVAILABILITY_ZONE = us-east-1a
>>>> VOLUMES = shhs
>>>>
>>>>
>>>> [cluster high-cpu-xl]
>>>> EXTENDS = small
>>>> NODE_IMAGE_ID = ami-a19e71c8
>>>> NODE_INSTANCE_TYPE = c1.xlarge
>>
>> _______________________________________________
>> Starcluster mailing list
>> Starcluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla -
http://enigmail.mozdev.org/
iEYEARECAAYFAkvR++gACgkQ4llAkMfDcrm4OgCeJ/nNH1Ehtv0o9Z53ajAWukuD
494AniEX6J+NWdlvUeCXU7LCIvHIfgM3
=AVJj
-----END PGP SIGNATURE-----
Received on Fri Apr 23 2010 - 15:58:35 EDT