Hi,
I also need information about the command line option for making a spot
request persistent. I am currently using:
starcluster start -o -b mybid mycluster
and in the AWS online GUI console, I see that this generates a one-time
spot request not a persistent one. I googled on AWS or starcluster to find
out how to change this after request or when I make the initial request,
thank you very much for your support,
Kind Regards,
Manal
On 19 May 2012 11:17, Manal Helal <manalorama_at_gmail.com> wrote:
> Hi Rayson,
>
> Thank you very much for the extended explanations and for sharing your
> experience. I had to start fast, and read only quick-guides, and first two
> pages of everything, and googling what I need only,
>
> thanks again,
>
> Kind Regards,
>
> Manal
>
>
> On 19 May 2012 02:47, Rayson Ho <raysonlogin_at_gmail.com> wrote:
>
>> Manal,
>>
>> Here's my spending back in Feb. I am also in the free tier, BTW.
>>
>> In that month, I used 27 hours of t1.micro & 1 hour of m1.small, thus
>> I got charged $ 0.09 ($0.085 rounded up!).
>>
>> Amazon EC2 running Linux/UNIX
>> $0.00 per Micro Instance (t1.micro) instance-hour (or
>> partial
>> hour) under monthly free tier 27 Hrs 0.00
>> $0.085 per Small Instance (m1.small) instance-hour (or
>> partial
>> hour) 1 Hr 0.09
>>
>> Amazon EC2 EBS
>> $0.00 per GB-month of provisioned storage under monthly
>> free
>> tier 0.398 GB-Mo 0.00
>> $0.00 per 1 million I/O requests under monthly free tier
>> 428,595 IOs 0.00
>>
>> Elastic IP Addresses
>> $0.00 per Elastic IP address remap - first 100 remaps /
>> month 1
>> Count 0.00
>>
>>
>> I am under the free EBS limit (it was then 10GB month, but the limit
>> got bumped up to 30 GB month recently). Note that you can use 60GB for
>> 15 days, then 0 GB for the rest of the month, and Amazon will not
>> charge you anything. I think Amazon has a very straightforward
>> pricing, and you don't need to worry about whether the EBS is used by
>> a t1.micro or a cg1.4xlarge or even just left there for backup...
>>
>> Basically, feel free to mix and match free & non-free EC2 resource.
>>
>> Rayson
>>
>> ================================
>> Open Grid Scheduler / Grid Engine
>> http://gridscheduler.sourceforge.net/
>>
>> Scalable Grid Engine Support Program
>> http://www.scalablelogic.com/
>>
>>
>>
>> On Thu, May 17, 2012 at 9:59 PM, Rayson Ho <raysonlogin_at_gmail.com> wrote:
>> > So you are in the "free tier" indeed! Keep in mind that you get 30 GB
>> > of free EBS storage, and if you are willing to pay, then you don't
>> > need to limit yourself to micro instances.
>> >
>> > For example, you can still use CG1 instances (which are not free even
>> > for those in the free tier) and you still get free 30GB of EBS
>> > storage. You basically can mix and match free & non-free stuff in AWS.
>> >
>> > Rayson
>> >
>> > ================================
>> > Open Grid Scheduler / Grid Engine
>> > http://gridscheduler.sourceforge.net/
>> >
>> > Scalable Grid Engine Support Program
>> > http://www.scalablelogic.com/
>> >
>> >
>> >
>> > On Thu, May 17, 2012 at 9:27 PM, Manal Helal <manalorama_at_gmail.com>
>> wrote:
>> >> Thank you very much Justin and Rayson for the explanation,
>> >>
>> >> I actually started the cluster Yesterday and learned a few issues by
>> try and
>> >> error, realising the already mounted EBS in the starcluster HVM AMI,
>> but
>> >> terminates with it and needs downloading the data. I also realised the
>> >> unfeasibility of the thought of downloading an image, updating it on my
>> >> machine, and then upload it after testing and everytime I need it. Also
>> >> using a spotnode cost less, so I can experiment with less worries, but
>> the
>> >> IO free-tier is small, and only tiny instances are in the free tier,
>> and I
>> >> need cg1.4xlarge to use the GPU Cluster,
>> >>
>> >> I tried the following steps:
>> >>
>> http://instantbadger.blogspot.com.au/2009/09/how-to-create-and-save-ami-image-from.html
>> >>
>> >> to bundle, upload and register the customised running AMI after my
>> updates,
>> >> and the bundle took time indeed, but this seams like the alternative
>> to the
>> >> scenario download/upload I was thinking of,
>> >>
>> >> thanks again for your help,
>> >>
>> >> Kind Regards,
>> >>
>> >>
>> >> On 18 May 2012 07:40, Rayson Ho <raysonlogin_at_gmail.com> wrote:
>> >>>
>> >>> On Thu, May 17, 2012 at 5:23 PM, Justin Riley <jtriley_at_mit.edu>
>> wrote:
>> >>> > If you're trying to find a way to store data without having to keep
>> the
>> >>> > cluster around there are much better approaches:
>> >>>
>> >>> Manal,
>> >>>
>> >>> Since you mentioned that you are new to AWS, you should be able to get
>> >>> the benefits of the free tier:
>> >>>
>> >>> http://aws.amazon.com/free/
>> >>>
>> >>> Basically, you can use 30 GB of EBS storage free of charge. On the
>> >>> other hand, if you pull data from AWS to your local harddrive
>> >>> everytime you are done with the instance, then you will need to pay
>> >>> for I/O & data transfer costs.
>> >>>
>> >>>
>> >>> Justin,
>> >>>
>> >>> I am trying to find the version of the NVIDIA Driver & CUDA toolkit
>> >>> installed on the CG1 AMIs, but instead of booting up an instance to
>> >>> find out what's there, is there a page that lists the additional
>> >>> software packages installed in the AMIs?? So far, I could only find:
>> >>>
>> >>> https://github.com/jtriley/StarCluster/issues/9
>> >>>
>> >>> Rayson
>> >>>
>> >>> ================================
>> >>> Open Grid Scheduler / Grid Engine
>> >>> http://gridscheduler.sourceforge.net/
>> >>>
>> >>> Scalable Grid Engine Support Program
>> >>> http://www.scalablelogic.com/
>> >>>
>> >>>
>> >>>
>> >>> >
>> >>> > 1. Use an external EBS volume with StarCluster and store your data
>> >>> > there. This will allow you to terminate the cluster and still
>> preserve
>> >>> > your important data on the EBS volume. See:
>> >>> >
>> >>> > http://web.mit.edu/star/cluster/docs/latest/manual/volumes.html
>> >>> >
>> >>> > 2. If you don't wish to use external EBS volumes at all you can
>> always
>> >>> > use starcluster's "get" command to manually download files from the
>> >>> > cluster before terminating, e.g.:
>> >>> >
>> >>> > $ starcluster get mycluster /path/to/results.tar.gz
>> >>> >
>> >>> > Similarly you can upload files using the "put" command:
>> >>> >
>> >>> > $ starcluster put mycluster /path/to/local/file /remote/path/
>> >>> >
>> >>> > See the following doc for more details:
>> >>> >
>> >>> > http://web.mit.edu/star/cluster/docs/latest/manual/putget.html
>> >>> >
>> >>> >> 4. After knowing how to download the image, I will need the
>> command
>> >>> >> to
>> >>> >> start a new cluster using the image I have stored on my local
>> disk.
>> >>> >> I
>> >>> >> think I have seen something to install from an image to the
>> cluster
>> >>> >
>> >>> > Launching a cluster using a locally stored disk image is not
>> supported.
>> >>> > The latency in doing this would be *insane*. Having to upload
>> multiple
>> >>> > GBs and register an AMI everytime would make starting a cluster
>> >>> > *extremely* time-consuming and error-prone.
>> >>> >
>> >>> > As mentioned above you're better off creating a new AMI for your
>> needs
>> >>> > and keeping the new AMI on Amazon. Then simply update your
>> StarCluster
>> >>> > config to use your new AMI. If you need to save data before
>> terminating
>> >>> > the cluster then either save your data to an external EBS volume or
>> copy
>> >>> > the data using the 'get' command as mentined above.
>> >>> >
>> >>> > Hope that helps,
>> >>> >
>> >>> > ~Justin
>> >>> >
>> >>> > _______________________________________________
>> >>> > StarCluster mailing list
>> >>> > StarCluster_at_mit.edu
>> >>> > http://mailman.mit.edu/mailman/listinfo/starcluster
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ==================================================
>> >>> Open Grid Scheduler - The Official Open Source Grid Engine
>> >>> http://gridscheduler.sourceforge.net/
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Kind Regards,
>> >>
>> >> Manal Helal
>> >>
>> >
>> >
>> >
>> > --
>> > ==================================================
>> > Open Grid Scheduler - The Official Open Source Grid Engine
>> > http://gridscheduler.sourceforge.net/
>>
>>
>>
>> --
>> ==================================================
>> Open Grid Scheduler - The Official Open Source Grid Engine
>> http://gridscheduler.sourceforge.net/
>>
>
>
>
>
--
Kind Regards,
Manal Helal
Received on Fri May 18 2012 - 21:24:47 EDT
This archive was generated by
hypermail 2.3.0.