StarCluster - Mailing List Archive

Re: access instance storage

From: Justin Riley <no email>
Date: Mon, 08 Aug 2011 14:01:13 -0400

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Sebastian,

Unfortunately this is not possible when creating a new HVM image using
the ebsimage command. This is a limitation imposed by EC2's
'create-image' API call. The 'create-image' call does not support
specifying a block device mapping that's different from the image host's
block device mapping. This is why you don't see the ephemeral storage on
your new 11.04 HVM image; the original alestic HVM didn't have it in the
block device mapping. To make matters worse the EC2 API documentation
specifically states that an HVM image must be created from an existing
HVM instance in order for the AMI to be HVM-compatible:

http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using_cluster_computing.html#using_cluster_computing_hvm_ami_running_instance

There may be a way around this though. It is possible to manually
specify a block device mapping when launching the instances. This would
allow StarCluster to override the default block device mapping specified
by a given AMI and always provide ephemeral storage even if the original
AMI creator did not include it when registering the image. I'll
experiment with this and see what I can come up with. I've created a
github issue to keep track of this:

https://github.com/jtriley/StarCluster/issues/46

Thanks for reporting,

~Justin

On 08/08/2011 09:30 AM, sebastian rooks wrote:
> Hello Justin,
>
> I would like to use the instance storage (2x 840G disks) that comes
> with the cluster compute instance type. This is again for an Ubuntu
> 11.04 hvm AMI.
>
> The scratch space is not mounted on /mnt: root_at_master:/mnt/sgeadmin#
> df -h Filesystem Size Used Avail Use% Mounted on
> /dev/sda1 7.9G 4.0G 3.5G 54% / none
> 12G 144K 12G 1% /dev none 12G 0 12G 0%
> /dev/shm none 12G 60K 12G 1% /var/run none
> 12G 0 12G 0% /var/lock /dev/xvdz1 20G 618M 19G
> 4% /home
>
> The block devices are indeed not available on my AMI:
>
> ec2-describe-images ami-ab5197c2 -K ~/.starcluster/pk-....pem -C
> ~/.starcluster/cert-....pem IMAGE ami-ab5197c2 11-04-hvm available
> private x86_64 machine ebs hvm BLOCKDEVICEMAPPING /dev/sda1
> snap-8f055aee 8
>
> Nor are they on Alestic's public one:
>
> ec2-describe-images ami-1cad5275 -K ~/.starcluster/pk-....pem -C
> ~/.starcluster/cert-....pem IMAGE ami-1cad5275
> 099720109477/hvm/ubuntu-images/ubuntu-natty-11.04-amd64-server-20110426
> 099720109477 available public x86_64 machine ebs hvm
> BLOCKDEVICEMAPPING /dev/sda1 snap-b1ad2dde 8
>
> Would it be possible to add the local instance storage during the
> AMI creation with ebsimage ?
> http://docs.amazonwebservices.com/AWSEC2/2011-01-01/UserGuide/index.html?Using_AddingDefaultLocalInstanceStorageToAMI.html
>
> Regards,
>
> Sebastian _______________________________________________ StarCluster
> mailing list StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5AJGkACgkQ4llAkMfDcrnmmQCdESKjN3GWw3edHajdDJb4ukc1
ZEMAnRetpFfu+ayuaat1wboKsyjOoVlh
=9BgL
-----END PGP SIGNATURE-----
Received on Mon Aug 08 2011 - 14:01:15 EDT
This archive was generated by hypermail 2.3.0.

Search:

Sort all by:

Date

Month

Thread

Author

Subject