Sorry for hijacking this thread but I have some additional questions
regarding spot instances and StarCluster.
Let's assume I start a cluster with four c1.xlarge spot instances. I
am able to split my compute jobs into really small chunks, each one
running for 5 minutes on one CPU. Now, if my spot instances go away
after a certain amount of time, hopefully lots of my small chunks are
already completed.
Has anyone experience with the qsub -r parameter to restart those jobs
affected by the spot instance shutdown?
Assuming all my initial spot instance nodes are gone, will adding new
nodes to the cluster restart the jobs?
Any idea how to (automatically) add new spot instances for a certain
price if the old ones were shut down? I assume this will not be done
automatically?
My idea is to basically let the master node run and if cheap spot
instances are available, request them, add them to the cluster and
compute until they are taken away. Then wait and repeat.
Thanks
Fabian
2012/8/29 Joydeep Sen Sarma <jsensarma_at_gmail.com>:
> afaik:
>
> - yes - spot instances can be taken away anytime (when, presumably,
> the bid price is not high enough)
>
> - from what i have seen - bidding at around the on-demand instance
> price (or maybe just a bit higher) would be good enough strategy to a)
> minimize the odds of having nodes taken away b) get some general
> reduction in cost (since most times the spot price is 30-40% lower
> than on-demand price)
>
> the higher you bid above the spot price - the less, obviously, the
> odds of the nodes getting taken away. at the same time - the bidding
> seems to be like a second-price auction - so higher bids, in general,
> don't translate into higher spends.
>
> there have been some random price spikes (hundreds of dollars in one
> case i think) - so the general advice is not bid some astronomical
> number.
>
>
> On Wed, Aug 29, 2012 at 3:00 PM, Ed Morris <ecm200_at_gmail.com> wrote:
>> Hi.
>>
>> I was wondering if anyone could help describe how Star Cluster behaves with
>> spot instances.
>>
>> If I specify a bid price at a certain level, and then the spot price exceeds
>> that whilst say, a parallel compute job is using those resources, what will
>> happen? Will the nodes be shut down as the bid price has been exceeded?
>>
>> Or is the bid price simply used to start the nodes when the price reduces to
>> a certain cost level or below?
>>
>> I am running parallel compute jobs that take a few days to complete a single
>> job, and hence termination of compute nodes during the job would cause
>> issues.
>>
>> However, I would very much like to gain access to the reduced costs of spot
>> prices, with the obvious chance that I could pay more if the spot price rose
>> above the fixed price.
>>
>> Any help would be greatly appreciated.
>>
>> Kind regards,
>>
>> Ed Morris.
>>
>> --
>> Ed Morris
>> ------------------------------------------------
>> email: ecm200_at_gmail.com
>> phone:
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Mon Sep 03 2012 - 15:44:53 EDT