Re: Spot Instance Market Price Spikiness: Ideas?

From: Justin Riley <no email>
Date: Thu, 16 Sep 2010 15:58:25 -0400

Hi Damian,

> Has anyone else noticed spikiness in the price for spot instances
> within the past two weeks.

Unfortunately I've only been using spot instances for development lately
so I'm not trying to keep them up for days at a time. I agree with
Austin your best bet is to up your bid. You'll still pay the same amount
(the current spot price) but you'll be more resilient to this spikiness.

> Do you know if there is any way to save the state of the cluster when
> a system halt signal is given by AWS?

Currently there is not, however, using EBS-backed instances would allow
you to save the instance's root partition and any data that was left.
You could then attach that volume to another instance later on to
recover the data.

I'm planning on adding EBS-backed AMIs to StarCluster and Fred Rotbart
has submitted patches that handle starting/stopping EBS-backed instances
including instances of the new cluster compute type (which you'll be
*very* interested in by the way given that you do lots of MPI). Stay tuned.

~Justin


On 09/14/2010 12:58 PM, Austin Godber wrote:
> Bid more. Are you really that sensitive to paying an extra $6/hr a few
> times a week? Isn't it worth avoiding a restart?
>
> I have definitely noticed a high frequency of insufficient capacity
> errors in us-east-1 recently.
>
> Austin
>
>
> On 9/14/10 9:39 AM, Damian Eads wrote:
>> Hi,
>>
>> Has anyone else noticed spikiness in the price for spot instances
>> within the past two weeks. I've been finding I'm lucky to get a 5 or
>> 10 node cluster up for more than two days, and in some cases, no more
>> than an hour. The market price for a c1.xlarge averages about
>> $0.25/hour but can quickly rise to $0.80 for a short period (<2
>> hours). Other instance types have also experienced similar spikiness
>> lately.
>>
>> Do you know if there is any way to save the state of the cluster when
>> a system halt signal is given by AWS?
>>
>> Just curious. Thank you in advance,
>>
>> Damian
>>
>> -----------------------------------------------------
>> Damian Eads Ph.D. Candidate
>> University of California Computer Science
>> 1156 High Street Machine Learning Lab, E2-489
>> Santa Cruz, CA 95064 http://www.soe.ucsc.edu/~eads
>>
>> Research Visitor, Machine Intelligence Lab
>> Department of Engineering, University of Cambridge
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Thu Sep 16 2010 - 15:58:32 EDT