Re: Updating public AMIs to have Ubuntu 14.04? Or how to install starcluster on AWS AMI?

From: Avner May <no email>
Date: Fri, 29 Jan 2016 11:58:14 -0500

I just wanted to give one final request for there being public Ubuntu 14.04
StarCluster AMIs (EBS and HVM-EBS), which are listed in the output of the
"starcluster listpublic", so that other people don't go through the painful
experience I had to go through.

What is the process for adding a new public AMI to StarCluster?

Sergio Mafra has pointed out to me that ami-38b99850 is available and has
StarCluster installed on Ubuntu 14.04.

On Thu, Jan 28, 2016 at 9:34 PM, Ed Gray <gray_ed_at_hotmail.com> wrote:

> Having worked with Jennifer, I can tell you she's a star.
>
>
>
> Sent from my Smartphone; please forgive any spelling and grammar mishaps.
>
>
> -------- Original message --------
> From: Avner May <avnermay_at_cs.columbia.edu>
> Date: 01/28/2016 6:13 PM (GMT-05:00)
> To: Don Morton <don.morton_at_borealscicomp.com>, Jennifer Staab <
> jstaab_at_cs.unc.edu>
> Cc: starcluster_at_mit.edu
> Subject: Re: [StarCluster] Updating public AMIs to have Ubuntu 14.04? Or
> how to install starcluster on AWS AMI?
>
> Hi all,
>
> I have carefully followed Jennifer Staab's instructions here
> <http://star.mit.edu/cluster/mlarchives/2545.html>, and things are
> working great (*** THANK YOU SO MUCH JENNIFER!! ***). One small hiccup I
> ran into is that at that link, one of the commands is long and is split
> into 2 lines ('apt-get install nfs-kernel-server nfs-common rpcbind
> libgssglue1 libnfsidmap2 libtirpc1 -y") -- if this is copy-pasted as 2
> lines, it will fail quietly when instance is launched, and it eventually
> leads to an error starting NFS on the master (see error message below).
>
> I also followed the instructions here
> <https://github.com/BVLC/caffe/wiki/Install-Caffe-on-EC2-from-scratch-(Ubuntu,-CUDA-7,-cuDNN)>
> in order to install NVIDIA drivers and CUDA (though I installed the latest
> versions, not the ones included in that post).
>
> I would recommend including a modified version of Jennifer's instructions
> directly linked from the primary StarCluster website in order to explain
> how to set-up a non-StarCluster AMI to work with StarCluster. This is
> super valuable information.
>
> Now everything is working!!!
> Avner
>
> ========================
> Starting NFS server on master
> !!! ERROR - Error occured while running plugin
> 'starcluster.clustersetup.DefaultClusterSetup':
> !!! ERROR - remote command 'source /etc/profile && /etc/init.d/nfs
> !!! ERROR - start' failed with status 127:
> !!! ERROR - bash: /etc/init.d/nfs: No such file or directory
>
> On Thu, Jan 28, 2016 at 3:01 PM, Don Morton <don.morton_at_borealscicomp.com>
> wrote:
>
> Hello,
>
> I faced this same issue a few months ago, and somehow succeeded after
> bumping into lots of walls (most of it based on my own ignorance). I
> haven't done anything with this since, and these notes are extremely rough
> (hopefully I didn't cuss too much!), but you are welcome to use what you
> need. Maybe someone will have the time to make something more formal, but
> I'm afraid it won't be me for some time now.
>
> All the best,
>
> Don
>
> ---
> Don Morton, Owner/Manager
> Boreal Scientific Computing LLC
> Fairbanks, Alaska USA
> http://www.borealscicomp.com/
> http://www.borealscicomp.com/Miscellaneous/MortonBio/
>
> On Thu, Jan 28, 2016 at 7:42 PM, Avner May <avnermay_at_cs.columbia.edu>
> wrote:
>
> I see that this has already been asked several times:
> http://star.mit.edu/cluster/mlarchives/2690.html,
> http://star.mit.edu/cluster/mlarchives/2545.html,
> https://www.youtube.com/watch?v=2RBupgpi_ec, etc.
>
> Which of these guides should I use? Why hasn't an Ubuntu 14.04 AMI been
> made public by the StarCluster development team yet? Are there plans to do
> this? Meanwhile, could someone make their AMI public?
>
> It seems there is widespread interest in updated AMIs (EBS and HVM-EBS
> versions).
>
> Thanks so much,
> Avner
>
> On Thu, Jan 28, 2016 at 2:30 PM, Avner May <avnermay_at_cs.columbia.edu>
> wrote:
>
> Hi,
>
> My goal is to have a StarCluster AMI with Ubuntu 14.04, CUDA 7.5, and the
> latest NVIDIA drivers (I want to use g2.2xlarge instance type).
>
> I have noticed that the public StarCluster AMIs are pretty outdated, with
> the most up-to-date AMIs using Ubuntu 13.04, which has already reached its "end
> of life
> <http://fridge.ubuntu.com/2014/01/28/ubuntu-13-04-raring-ringtail-end-of-life-reached-on-january-27-2014/>".
> Could an Ubuntu 14.04 + 64-bit + HVM-EBS public AMI be created, with CUDA
> 7.5 and the latest NVIDIA driver
> <http://www.nvidia.com/download/driverResults.aspx/97645/en-us> installed?
> This would be incredibly useful. I am having a very hard time
> upgrading ami-6b211202 for my needs (I have run into issues updating
> ubuntu, as well as installing NVIDIA drivers and CUDA).
>
> Here
> <http://tleyden.github.io/blog/2014/10/25/cuda-6-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/>
> are instructions on how to install NVIDIA drivers and CUDA on Ubuntu 14.04
> AMI (ami-9eaa1cf6). So another option which would satisfy my needs would
> be if you simply gave me instructions on how to install StarCluster on this
> AMI.
>
> Any advice on the easiest way to get a working StarCluster AMI, with an up
> to date version of Ubuntu, would be great.
>
> Thanks!
> Avner
>
>
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
>
>
Received on Fri Jan 29 2016 - 11:58:37 EST