StarCluster - Mailing List Archive

Re: access instance storage

From: Justin Riley <no email>
Date: Mon, 08 Aug 2011 14:35:16 -0400

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Also, for your immediate needs, you can launch an instance of the
Alestic HVM 11.04 AMI manually and specify the block mapping including
the ephemeral storage device(s). Using ebsimage on an instance launched
with a custom block device mapping should create the new AMI with an
identical block device mapping. Let me know if you need help with this.
Basically the launch command should look something like:

ec2-run-instances <ami id> -k <your keypair name> --block-device-mapping
'/dev/sdb1=ephemeral0' --block-device-mapping '/dev/sdc1=ephemeral1'
- --block-device-mapping '/dev/sdd1=ephemeral2' --block-device-mapping
'/dev/sde1=ephemeral3'

~Justin

On 08/08/2011 02:01 PM, Justin Riley wrote:
> Hi Sebastian,
>
> Unfortunately this is not possible when creating a new HVM image
> using the ebsimage command. This is a limitation imposed by EC2's
> 'create-image' API call. The 'create-image' call does not support
> specifying a block device mapping that's different from the image
> host's block device mapping. This is why you don't see the ephemeral
> storage on your new 11.04 HVM image; the original alestic HVM didn't
> have it in the block device mapping. To make matters worse the EC2
> API documentation specifically states that an HVM image must be
> created from an existing HVM instance in order for the AMI to be
> HVM-compatible:
>
> http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/using_cluster_computing.html#using_cluster_computing_hvm_ami_running_instance
>
> There may be a way around this though. It is possible to manually
> specify a block device mapping when launching the instances. This
> would allow StarCluster to override the default block device mapping
> specified by a given AMI and always provide ephemeral storage even if
> the original AMI creator did not include it when registering the
> image. I'll experiment with this and see what I can come up with.
> I've created a github issue to keep track of this:
>
> https://github.com/jtriley/StarCluster/issues/46
>
> Thanks for reporting,
>
> ~Justin
>
> On 08/08/2011 09:30 AM, sebastian rooks wrote:
>> Hello Justin,
>
>> I would like to use the instance storage (2x 840G disks) that comes
>> with the cluster compute instance type. This is again for an
>> Ubuntu 11.04 hvm AMI.
>
>> The scratch space is not mounted on /mnt:
>> root_at_master:/mnt/sgeadmin# df -h Filesystem Size Used
>> Avail Use% Mounted on /dev/sda1 7.9G 4.0G 3.5G 54% /
>> none 12G 144K 12G 1% /dev none 12G 0
>> 12G 0% /dev/shm none 12G 60K 12G 1%
>> /var/run none 12G 0 12G 0% /var/lock /dev/xvdz1
>> 20G 618M 19G 4% /home
>
>> The block devices are indeed not available on my AMI:
>
>> ec2-describe-images ami-ab5197c2 -K ~/.starcluster/pk-....pem -C
>> ~/.starcluster/cert-....pem IMAGE ami-ab5197c2 11-04-hvm available
>> private x86_64 machine ebs hvm BLOCKDEVICEMAPPING /dev/sda1
>> snap-8f055aee 8
>
>> Nor are they on Alestic's public one:
>
>> ec2-describe-images ami-1cad5275 -K ~/.starcluster/pk-....pem -C
>> ~/.starcluster/cert-....pem IMAGE ami-1cad5275
>> 099720109477/hvm/ubuntu-images/ubuntu-natty-11.04-amd64-server-20110426
>>
>>
099720109477 available public x86_64 machine ebs hvm
>> BLOCKDEVICEMAPPING /dev/sda1 snap-b1ad2dde 8
>
>> Would it be possible to add the local instance storage during the
>> AMI creation with ebsimage ?
>> http://docs.amazonwebservices.com/AWSEC2/2011-01-01/UserGuide/index.html?Using_AddingDefaultLocalInstanceStorageToAMI.html
>
>> Regards,
>
>> Sebastian _______________________________________________
>> StarCluster mailing list StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>
> _______________________________________________ StarCluster mailing
> list StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5ALGQACgkQ4llAkMfDcrkdkQCfafIHF1jdFFG5XlIz4ogsqdMH
1AkAn3pT4Sm1b4jA+Bdh4EIiDxDboKDE
=ZK3g
-----END PGP SIGNATURE-----
Received on Mon Aug 08 2011 - 14:35:32 EDT
This archive was generated by hypermail 2.3.0.

Search:

Sort all by:

Date

Month

Thread

Author

Subject