Hi Ryan,
I experimented with addnode a while back:
1) Couldn't add an on-demand instance to a cluster initiated w/spot-bid
instances.
2) *Could* add a spot-bid instance to a cluster initiated w/on-demand
instances.
3) addnode command in any case will add whatever instance type you specify
with "-I", so you can run heterogeneously amongst the compute hosts (with
each instance potentially having a different bid price).
Cheers,
Lyn
---------- Forwarded message ----------
From: Ryan Golhar <ngsbioinformatics_at_gmail.com>
To: starcluster_at_mit.edu
Cc:
Date: Thu, 24 Jan 2013 12:39:47 -0500
Subject: [StarCluster] adding nodes to an existing cluster of a different
instance type
Is it possible to add a node to a running cluster that is different from
the instance type in the config file? I didn't see anything in the docs
---------- Forwarded message ----------
From: "MacMullan, Hugh" <hughmac_at_wharton.upenn.edu>
To: "'Ed Morris'" <ecm200_at_gmail.com>
Cc: "'starcluster_at_mit.edu'" <starcluster_at_mit.edu>
Date: Thu, 24 Jan 2013 17:40:56 +0000
Subject: Re: [StarCluster] Amazon AMI 64-bit Ubuntu 11 HVM-EBS for
eu-west-1 region
Hi Ed:****
** **
Woohoo, great news!****
** **
Lincoln Stein, maintainer of the VM::EC2 Perl module, has added HVM copy
functionality to his (already pretty awesome) module! The module includes
the Perl script migrate-ebs-image.pl, which makes copies a breeze.****
** **
I just tested it with a copy of a Scientific Linux 6.3 HVM StarCluster
image that I use frequently. Here’s the output:****
** **
# migrate-ebs-image.pl --from us-east-1 --to eu-west-1 ami-XXXXXXXXX****
[info] Scanning for existing staging servers and volumes in
http://ec2.us-east-1.amazonaws.com.****
[info] Scanning for existing staging servers and volumes in
http://ec2.eu-west-1.amazonaws.com.****
[info] Gathering information about image ami-XXXXXXXXX.****
[info] Copying EBS volumes attached to this image (this may take a long
time).****
[info] Copying snapshot snap-XXXXXXXX from us-east-1 to eu-west-1...**
**
[info] ...new snapshot = snap-XXXXXXXXXX; status = pending****
[info] Waiting for all snapshots to complete. This may take a long time.*
***
An image named 'starcluster-base-scientific-linux-6.3-x86_64-hvm-005'
already exists in destination region. Renamed to
'starcluster-base-scientific-linux-6.3-x86_64-hvm-005-d88a4dfe' *ß** I
REALLY LIKE THIS!!!*****
[info] Registering snapshot in destination with the equivalent of:****
[info] ec2-register -n
'starcluster-base-scientific-linux-6.3-x86_64-hvm-005-XXXXX' -d 'Image
created _at_ 201301202215' -a x86_64 --kernel '' --ramdisk ''
--root-device-name /dev/sda1 -b /dev/sda1=snap-XXXXXXXXXXX:false:standard***
*
[info] Searching for a suitable HVM image in destination region****
[info] ...Found ami-011b1975 (suse-sles-11-sp2-hvm-v1.01.x86_64)****
[info] Launching an HVM staging server in the target region.
Heuristically choosing instance type of 'cc2.8xlarge' for this type of HVM..
****
[info] Waiting for instance to become ready.****
[info] Stopping instance temporarily to swap root volumes.****
[info] Detaching original root volume...****
[info] Creating and attaching new root volume..****
[info] Creating image in destination region...****
[info] Terminating the staging server****
New snapshot is now located in eu-west-1 under ami-XXXXXXXXX.****
** **
No muss, no fuss. As you can see, it has to launch a cc2.8xlarge to do the
copy, so it’ll cost you to do the transfer. It’s currently in the
hvm-migrate alpha GIT tree:****
** **
https://github.com/lstein/LibVM-EC2-Perl/tree/hvm-migrate****
** **
He’ll be publishing to CPAN today or tomorrow after Windows AMI testing. It
will be VM::EC2 1.23 on CPAN.****
** **
So there you go.****
** **
-Hugh****
** **
** **
*From:* Ed Morris [mailto:ecm200_at_gmail.com]
*Sent:* Friday, January 18, 2013 12:43 PM
*To:* MacMullan, Hugh
*Cc:* starcluster_at_mit.edu
*Subject:* Re: [StarCluster] Amazon AMI 64-bit Ubuntu 11 HVM-EBS for
eu-west-1 region****
** **
** **
Hi Hugh.****
** **
Thanks for that, much appreciated.****
** **
I tried using the "snapshot" route, which allows you to transfer snapshots
between regions through the AWS web console (AWS documentation
here<
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-copy-snapshot.html>),
but so far I have had no luck with regards to that. I started an instance
with the US-EAST-1 Ubuntu 11.10 x86_64 HVM AMI (not through Star Cluster),
and then created a snap shot of the root volume. I then transferred the
snapshot to EU-WEST-1 region and converted that into an AMI. One issue I
had, and have still not solved, was the "kernel ID" specification which was
needed during the creation of the AMI from the snap shot. I could not find
the kernel ID for the AMI in question, even though other flavours of Star
Cluster AMI's did have associated Kernel IDs. I came unstuck when I tried
to execute the AMI that I had transferred from the US-EAST-1 region to
EU-WEST-1 region, and I was told by StarCluster that the AMI was not HVM,
and therefore could not be executed on the CC2.8xlarge instance type. I am
positive that I am using the correct AMI which is HVM supported, as this is
currently the only HVM supported AMI that's available, and it's only
available in US-EAST-1.****
** **
I plan to try something slightly different, by creating a custom AMI using
the StarCluster tools, and then creating a snap shot from that AMI, and
transferring that. But again I may come up against the Kernel ID issue,
which I still don't know how to solve.****
** **
However some good news, I have read in a number of places on the web that
Amazon are developing a tool for transferring AMI's the same way you can
now transfer snap shots, and can be undertaken through the AWS console. I
haven't managed to find when this is expected to be implemented though.****
** **
There is also this that I found (Ylastic <
http://ylastic.com/features.html>),
which amongst a lot of things, appears to suggest that it can migrate EBS
AMI's from region to region.****
** **
** **
On 18 January 2013 18:16, MacMullan, Hugh <hughmac_at_wharton.upenn.edu> wrote:
****
Hi Ed:****
****
I have no EU AMIs, but I’ve been curious about the second question you
have: what’s the ‘easy’ way to transfer AMIs? I poked around a bit today,
and it looks like ‘migrate-ebs-image.pl’, part of the VM::EC2 Perl module,
is a good solution … *except* that it converts my HVM AMIs to PV. Argh! J***
*
****
I’ve got a query in with the developer (Lincoln Stein) about HVM copies.
I’ll let you know what I discover.****
****
That said, if others have other *easy* ways to transfer AMIs … I’m all ears
too!****
****
-Hugh****
****
****
*From:* starcluster-bounces_at_mit.edu [mailto:starcluster-bounces_at_mit.edu] *On
Behalf Of *Ed Morris
*Sent:* Friday, January 18, 2013 5:26 AM
*To:* starcluster_at_mit.edu
*Subject:* [StarCluster] Amazon AMI 64-bit Ubuntu 11 HVM-EBS for eu-west-1
region****
****
Hi.****
****
Does anyone know if there is an AMI available using HVM version of Ubuntu
for the cc8.xlarge compute node for starcluster in the eu-west-1 region?****
****
I have been quite happily using the AMI ami-4583572c us-east-1
starcluster-base-ubuntu-11.10-x86_64-hvm (HVM-EBS) on the us-east region,
but couldn't find any HVM versions in the eu region.****
****
Any ideas? Can the HVM-EBS be easily transferred to the eu region?****
****
Ed.
****
****
--
Ed Morris
------------------------------------------------
email: ecm200_at_gmail.com
phone: ****
****
** **
--
Ed Morris
------------------------------------------------
email: ecm200_at_gmail.com
phone: ****
_______________________________________________
StarCluster mailing list
StarCluster_at_mit.edu
http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Thu Jan 24 2013 - 13:00:03 EST