Re: Question about star cluster AMIs

From: Avner May <no email>
Date: Thu, 22 Jan 2015 14:39:13 -0500

Thanks so much Jacob.

I am considering the following options:
Option 1:
1) Launching an "imagehost" with an AWS AMI that already has CUDA on it (
https://aws.amazon.com/marketplace/pp/B00FYCDDTE)
2) Installing star cluster on it (
http://star.mit.edu/cluster/docs/latest/installation.html)
3) Create an EBS-backed AMI from this:
http://star.mit.edu/cluster/docs/latest/manual/create_new_ami.html

Option 2:
1) Launching an "imagehost" with one of the public starcluster AMIs (eg, [11]
ami-12b6477b us-east-1 starcluster-base-centos-5.4-x86_64-ebs-hvm-gpu-rc2)
2) Updating its cuda version (
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_cluster_computing.html#install-nvidia-driver
)
3) Create an EBS-backed AMI from this:
http://star.mit.edu/cluster/docs/latest/manual/create_new_ami.html

Would both of these methods work? Is one preferable to the other? Is
option #1, step 2, the correct way of installing star cluster on an AMI for
it to work correctly with star cluster?

Thanks,
Avner

On Thu, Jan 22, 2015 at 1:32 PM, Jacob Barhak <jacob.barhak_at_gmail.com>
wrote:

> Hi Avner,
>
> This is a quick answer, until one of the specialist returns with a better
> one.
>
> First, you have to make sure that the instance you are renting from Amazon
> will be compatible and have the GPU hardware you need.
>
> Then, the safest thing is to run the AMI on your selected instance and see
> if your software runs outside StarCluster using the Amazon interfaces. If
> you see you have all the software and hardware you need, move to configure
> StarCluster to create a cluster with your AMI.
>
> Each AMI has many software tools on it, so chances are that unless
> specific software is advertised you need to check if it exists by running
> it. Yet note that AMI and instance are separate definitions.
>
> I hope this gives you something to work with.
>
> Jacob
>
> On Thu, Jan 22, 2015 at 12:04 PM, Avner May <avnermay_at_cs.columbia.edu>
> wrote:
>
>> Hi all,
>>
>> I'm a new star cluster user, and a new member to this list. Not sure
>> what the etiquette is for emailing the list.
>>
>> I have the following question: When I perform "starcluster listpublic",
>> I see a list of AMI's (see command output below). However, I have no idea
>> how each of these AMIs is configured. For example, I need GPU support for
>> my work, so I would like my AMI to have CUDA pre-installed, and for it to
>> support being launched with the AWS GPU instances (eg, "g2.2xlarge").
>> Where can I get detailed information about each of these AMI's??
>>
>> Thanks so much,
>> Avner
>>
>> =============================
>> C:\Windows\system32>starcluster listpublic
>> StarCluster - (http://star.mit.edu/cluster) (v. 0.95.6)
>> Software Tools for Academics and Researchers (STAR)
>> Please submit bug reports to starcluster_at_mit.edu
>>
>> >>> Listing all public StarCluster images...
>>
>> 32bit Images:
>> -------------
>> [0] ami-9bf9c9f2 us-east-1 starcluster-base-ubuntu-13.04-x86 (EBS)
>> [1] ami-7c5c3915 us-east-1 starcluster-base-ubuntu-12.04-x86 (EBS)
>> [2] ami-899d49e0 us-east-1 starcluster-base-ubuntu-11.10-x86 (EBS)
>> [3] ami-8cf913e5 us-east-1 starcluster-base-ubuntu-10.04-x86-rc3
>> [4] ami-d1c42db8 us-east-1 starcluster-base-ubuntu-9.10-x86-rc8
>>
>> 64bit Images:
>> --------------
>> [0] ami-3393a45a us-east-1 starcluster-base-ubuntu-13.04-x86_64 (EBS)
>> [1] ami-6b211202 us-east-1 starcluster-base-ubuntu-13.04-x86_64-hvm
>> (HVM-EBS)
>> [2] ami-52a0c53b us-east-1 starcluster-base-ubuntu-12.04-x86_64-hvm
>> (HVM-EBS)
>> [3] ami-765b3e1f us-east-1 starcluster-base-ubuntu-12.04-x86_64 (EBS)
>> [4] ami-4583572c us-east-1 starcluster-base-ubuntu-11.10-x86_64-hvm
>> (HVM-EBS)
>> [5] ami-999d49f0 us-east-1 starcluster-base-ubuntu-11.10-x86_64 (EBS)
>> [6] ami-0af31963 us-east-1 starcluster-base-ubuntu-10.04-x86_64-rc1
>> [7] ami-2faa7346 us-east-1
>> starcluster-base-ubuntu-10.04-x86_64-qiime-1.4.0 (EBS)
>> [8] ami-8852a0e1 us-east-1 starcluster-base-ubuntu-10.04-x86_64-hadoop
>> [9] ami-a5c42dcc us-east-1 starcluster-base-ubuntu-9.10-x86_64-rc4
>> [10] ami-06a75a6f us-east-1
>> starcluster-base-centos-5.4-x86_64-ebs-hvm-gpu-hadoop-rc2 (HVM-EBS)
>> [11] ami-12b6477b us-east-1
>> starcluster-base-centos-5.4-x86_64-ebs-hvm-gpu-rc2 (HVM-EBS)
>>
>> total images: 17
>>
>>
>>
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
>>
>
Received on Thu Jan 22 2015 - 14:39:36 EST