Re: advice about using starcluster with ubuntu
Hi Jennifer,
Thank you very much for this guide to getting Ubuntu 14.04 running on
Starcluster, it has been very helpful. I had one quick question for
you. You mention that you were able to edit the starcluster code base
to allow for the provision of C4 instance types. Could you provide a
bit of specific help on exactly how to do that? I have poked around a
bit in the code base (0.95.6), and haven't been able to determine
where I would update the instance types / how I would then make sure
that they run properly. Any help you could give would be greatly
appreciated.
Thanks in advance,
Josh
On Thu, Feb 19, 2015 at 8:38 PM, <starcluster-request_at_mit.edu> wrote:
> Send StarCluster mailing list submissions to
> starcluster_at_mit.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/starcluster
> or, via email, send a message with subject or body 'help' to
> starcluster-request_at_mit.edu
>
> You can reach the person managing the list at
> starcluster-owner_at_mit.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of StarCluster digest..."
>
>
> Today's Topics:
>
> 1. Re: advice about using starcluster with ubuntu 14.04
> (Jennifer Staab)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 19 Feb 2015 15:38:03 -0500
> From: Jennifer Staab <jstaab_at_cs.unc.edu>
> Subject: Re: [StarCluster] advice about using starcluster with ubuntu
> 14.04
> To: Cedar McKay <cmckay_at_uw.edu>, "starcluster_at_mit.edu"
> <starcluster_at_mit.edu>
> Message-ID: <54E649AB.7000209_at_cs.unc.edu>
> Content-Type: text/plain; charset="windows-1252"
>
> Here's some help with creating an Ubuntu 14.04 AMI that works as a
> Starcluster node(master/worker). Below are directions for creating your
> own Ubuntu 14.04 LTS AMI. I didn't create a public AMI because I didn't
> want to step on the toes of the developers and I didn't want to be
> responsible for sanitizing private information off of an AMI that is
> made publically available. I also give the disclaimer that these
> directions "appear" to work to fix all issues with NFS and portmapper --
> but I only did light testing (e.g. start/terminate clusters and add &
> remove nodes - SGE & NFS seem to work as they should). Also the fixes I
> mention below I collected based on my own experience and others
> contributions to the mailing list.
>
> The portmapper issue isn't how I solved the issue on the version of
> Starcluster that I use in production because I don't have a problem
> editing to make the fix because there are some other tweaks I also edit
> the code for (like adding new C4 instance types in). That said it seems
> that the fix that creates a file /etc/init.d/portmap with "exit 0" seems
> to work -- my guess is the calls to portmap aren't really needed. The
> initial part of the script runs update and installs updates to the
> distribution, then it installs nfs-server and copies the init.d file to
> nfs as it is called by Starcluster scripts. Python is installed and
> then SGE is downloaded, unzipped and renamed to /sge6-fresh so
> Starcluster scripts work as advertised.
>
> How to make your own Ubuntu 14.04 LTS AMI:
> ---------------------------------------
> 1) Find your AMI look here <http://cloud-images.ubuntu.com/locator/ec2/>
> for all Ubuntu 14.04 LTS AMIs for your region -- grab the AMI ID for my
> example I used AMI ID: ami-36f8b15e -- its us-east-1, 14.04LTS, 64bit,
> ebs-ssd
> -- make sure to select and EBS backed EC2 AMI (not instance-store)
>
> 2) a) Login to AWS Console -- click on "launch an instance"
> b) select "Community AMIs" and put the AMI id from step one in to
> find it (here I used "ami-36f8b15e"), once it is found
> click "Select"
> c) Select instance type you want (choose something small because we
> are just creating an AMI to use) - click NEXT
> d) For "configure instance details" section -- make sure the
> instance is placed in your default VPC -- just so you CAN access it if
> need be
> e) Under the "Advanced Details" section click the triangle/arrow so
> that it expands. Paste in the following in the "User data" section. It
> runs the bash script that will install needed packages for
> making Starcluster work along with fixing issues with NFS and portmapper.
> This ASSUMES you are not using the StarCluster master node as
> StarCluster "master" AND StarCluster "controller". If you want to do
> this add "easy_install StarCluster" (no quotes included) after
> the last line "chmod +x /etc/init.d/portmap". Adding this line will
> install StarCluster on the EC2 you are launching. After you
> have pasted in the script in "User data" - click NEXT
> #!/bin/bash
> apt-get update
> apt-get dist-upgrade -y
> apt-get install nfs-kernel-server nfs-common rpcbind libgssglue1
> libnfsidmap2 libtirpc1 -y
> cp /etc/init.d/nfs-kernel-server /etc/init.d/nfs
> apt-get install sysv-rc-conf -y
> sysv-rc-conf nfs on
> /etc/init.d/nfs restart
> apt-get install python-setuptools python-dev -y
> cd /opt/
> wget http://dl.dropbox.com/u/47200624/respin/ge2011.11.tar.gz
> gunzip < ge2011.11.tar.gz | tar xvf -
> mv ge2011.11/ sge6-fresh/
> cd /
> echo 'exit 0' > /etc/init.d/portmap
> chmod +x /etc/init.d/portmap
>
> f) Click Next - so that you are in the "Tag Instance" section and
> Name your instance. I used "Ubuntu 14.04 Starcluster ready"
> g) Click Next - so you are in "Configure Security Group" section --
> here either select a pre-existing Security Group that you can
> SSH into -- OR -- Create a new security group that allows you
> to SSH into it from "My IP" (which is your IP).
> h) Click Review and Launch
> i) Click Launch
> j) Select an existing Keypair you have access too (so you can SSH
> into the instance) and click the check box and click "Launch
> Instances" OR create a new keypair, download that keypair and
> then click "Launch Instances".
>
> 3) Wait about 5-10 minutes and "Status Checks" should change from
> "Initializing" to " 2/2 checks passed" -- You can login to the instance
> and make sure issuing the command
> "sudo vi /etc/init.d/portmap" opens the file and shows the command
> "exit 0". This means the script completed and you can logout.
>
> 4) Stop the running instance by selecting "Actions - Instance State - Stop"
>
> 5) Select "Actions - Image - Create Image" to create the AMI -- name it
> something like "Ubuntu 14.04 Starcluster node AMI" and then select
> "Create Image" You will see "Create Image request received. View
> pending image ami-XXXX" note that AMI ID -- and that's the AMI ID you
> will use to create your Ubuntu 14.04 Starcluster master and worker nodes.
>
> 6) Once the AMI ID ami-XXXX - has Status "available" you can terminate
> the EC2 Instance we launched with the User data script because it is no
> longer needed. Use that AMI ID ami-XXXX in
> the StarCluster config file AND/OR when you want to launch master
> and worker nodes that are Ubuntu 14.04 LTS.
>
> Potential Issues:
> ---------------
> Step 1: DON'T choose "instance store" -- this isn't an EBS backed EC2 so
> when it is stopped all adjustments to OS are lost. Use something that
> says "ebs" in the instance type.
> ebs = magnetic disk, ebs-ssd = the general purpose ssd storage,
> ebs-ios = the Provisioned IOPS ssd storage.
> Step 3: WAIT at least 5-10 minutes before stopping the EC2 Instance --
> also it is really good idea to login to the EC2 and check that
> /etc/init.d/portmap says "exit 0" so you
> know that the script ran.
> Virtualization type issue -- Specifically your AMI is PV or HVM and you
> need the other type.
> 1) Go to Snapshots -- locate snapshot of AMI ID ami-XXX - note the
> "Description" has AMI ID in its text. Select that snapshot.
> 2) Select "Actions - Create Image" -- note in "Virtualization type"
> section you can select "Paravirtual" or "Hardware-assisted
> virtualization" -- so select virtualization type you
> need, Name the AMI, and select the other features you want then
> click "Create". This creates the AMI of the opposite virtualization type
> without any hassle. Using the
> snapshot to create and AMI of the virualization of your choice
> is WAY EASIER than the method I had used previously.
>
> Good Luck!
> -Jennifer
>
>
> On 2/16/15 12:27 PM, Cedar McKay wrote:
>> Some people, like Jennifer, have managed to get 14.04 AMIs working. It would be a real service if those people could share them.
>>
>> Also, does anyone have a reasonable explanation for why the official version is so slow in coming? Is there some sort of issue?
>>
>> Thanks,
>> Cedar
>>
>>
>>
>>> On Jan 29, 2015, at 6:41 AM, Pablo Escobar<pescobar001_at_gmail.com> wrote:
>>>
>>> Hi Jennifer
>>>
>>> thank you very much for your help. It's really useful.
>>>
>>> regards,
>>> Pablo.
>>>
>>> 2015-01-28 19:10 GMT+01:00 Jennifer Staab<jstaab_at_cs.unc.edu>:
>>> I installed Starcluster on Ubuntu 14.04 LTS and it is working fine for me. I didn't use the guide you mentioned, just a little guidance from emails on mailing list between Jacob and Hugh regarding installation (see here) and some sleuthing to get it to work. Hugh also provides a guideline/cookbook for creating new Starcluster AMI's here on the mailing list and said that just upgrading the existing AMI for 12.04 to 14.04 worked for him. To me that seems like the easiest way to handle this whole issue is to use existing Ubuntu 12.04 Starcluster AMI and just upgrade it to 14.04. I wasn't able to do this, so below is what worked for me. I am sure there are other ways to create a new Ubuntu 14.04 Starcluster AMI.
>
>>> --------------------------------------------------------------------------------------
>>> 1) Locate the AMI ID of the latest version of Ubuntu 14.04 for your region (see here). Start up your Ubuntu 14.04 AMI -- login and run update and apply all the security and distribution updates.
>>>
>>> 2) Install packages "python-dev" and "python-setuptools" -- You are welcome to install other packages, but those are the only ones needed if to do the Starcluster install. A list of all python packages can be found here .
>>>
>>> 3) You need to install NFS server so install package "nfs-kernel-server". Installing this package should install its dependencies (for me that included: nfs-common rpcbind libgssglue1 libnfsidmap2 libtirpc1). One issue I found was that in previous versions the startup script located in /etc/init.d/ was named "nfs" this installed version it's named "nfs-kernel-server". The easiest fix is to create a symbolic link to "/etc/init.d/nfs" (ln -s /etc/init.d/nfs-kernel-server /etc/init.d/nfs). Test by issuing the command "/etc/init.d/nfs restart" if it restarts then all is well.
>>>
>>> 4) Install Starcluster with "easy_install Starcluster"
>>>
>>> 5) You need to get a copy of the Open Grid Scheduler and place it in "/opt/sge6-fresh/" -- this is what Hugh and Jacob discuss in their emails on the mailing list. The easiest way I was able to do this was to use a running Ubuntu 12.04 Starcluster AMI (ami-765b3e1f) and copy "/opt/sge-fresh" from the 12.04 version to my Ubuntu 14.04 version. There are other ways to do this see their emails here.
>>>
>>> 6) Ubuntu 14.04 replaces portmap with rpcbind -- so this was an issue for me because the Starcluster software expects "portmap" to exist. So this is where I had to "edit" the code a bit.There are other ways to deal with the portmap issue, look here for an alternative solution. This is just how I handled the issue: locate the "node.py" portion of the code in the installed StarCluster version, mine was in "/usr/local/lib/python2.7/dist-packages/StarclusterXXX.egg/starcluster/node.py" On line 731 in the function "mount_nfs_shares()" you see line "self.ssh.execute('/etc/init.d/portmap start')". This won't work on Ubuntu 14.04 since "portmap" was replaced with "rpcbind". One way to fix this issue without breaking the code for other OS versions still using portmap is by replacing "self.ssh.execute('/etc/init.d/portmap start')" with a "try/except" statement so that if portmap doesn't work it calls rpcbind command instead; thus, handling the issue with portmap on Ubuntu 14.04. !
> B!
>> elow is how I changed the code in node.py:
>>> FIND line 731:
>>> self.ssh.execute('/etc/init.d/portmap start')
>>> REPLACE with:
>>> try:
>>> self.ssh.execute('/etc/init.d/portmap start')
>>> except:
>>> self.ssh.execute('service rpcbind restart')
>>>
>>> 7) Stop this instance and take an AMI of it. Use that AMI for nodes on your Starcluster clusters. If it bothers you to have Starcluster installed on the AMI you are using in your Starcluster clusters skip steps 2 and 4 BUT be certain to edit the version of Starcluster software you are using to create your Ubuntu 14.04 clusters to deal with portmap/rpcbind issue as described in step 6.
>>>
>>> A few things to note:
>>>
>>> 1) I only use starcluster commands to create clusters, terminate them, addnodes and removenodes (on-demand & spots) -- and this is what has been working for me. I haven't tested the other functionality, so there could be other issues like the portmap one that I don't know about. For the other functionality (like stopping instances, saving AMIs, etc), I use AWS directly to perform those functions.
>>> 2) The version of StarCluster that I used was 0.95.6 -- as available via easy_install. These issues (or ones similar to them) have been discussed and a few fixed in some of the development branches of Starcluster available on github see here and here.
>>> 3) I had some issue installing other software, but I am guessing most of that has been resolved now that it has been so long since the initial release of Ubuntu 14.04.
>>>
>>> Good Luck,
>>> Jennifer
>>>
>>>
>>> On 1/27/15 3:59 PM, Pablo Escobar wrote:
>>>> Hi all,
>>>>
>>>> I plan to use starcluster with ubuntu14.04. I will adapt the ami following this guide
>>>> https://www.youtube.com/watch?v=2RBupgpi_ec
>>>>
>>>> Is anyone in the list using starcluster with ubuntu 14.04 in production? Is it working fine or is there any drawback? is there any problem I should be aware before start using starcluster with ubuntu14.04?
>>>>
>>>> I know this has been asked before, but is there any plan to release a ubuntu14.04 official ami?
>>>>
>>>> thanks in advance for any advice.
>>>>
>>>> best regards,
>>>> Pablo.
>>>>
>>>>
>>>> _______________________________________________
>>>> StarCluster mailing list
>>>>
>>>> StarCluster_at_mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>> _______________________________________________
>>> StarCluster mailing list
>>> StarCluster_at_mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mailman.mit.edu/pipermail/starcluster/attachments/20150219/01de1500/attachment.htm
>
> ------------------------------
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
> End of StarCluster Digest, Vol 66, Issue 6
> ******************************************
--
Joshua White
jpwhite_at_gmail.com
+44 (0) 7932 919641
Received on Mon Feb 23 2015 - 18:14:04 EST
This archive was generated by
hypermail 2.3.0.