Here's some help with creating an Ubuntu 14.04 AMI that works as a
Starcluster node(master/worker). Below are directions for creating your
own Ubuntu 14.04 LTS AMI. I didn't create a public AMI because I didn't
want to step on the toes of the developers and I didn't want to be
responsible for sanitizing private information off of an AMI that is
made publically available. I also give the disclaimer that these
directions "appear" to work to fix all issues with NFS and portmapper --
but I only did light testing (e.g. start/terminate clusters and add &
remove nodes - SGE & NFS seem to work as they should). Also the fixes I
mention below I collected based on my own experience and others
contributions to the mailing list.
The portmapper issue isn't how I solved the issue on the version of
Starcluster that I use in production because I don't have a problem
editing to make the fix because there are some other tweaks I also edit
the code for (like adding new C4 instance types in). That said it seems
that the fix that creates a file /etc/init.d/portmap with "exit 0" seems
to work -- my guess is the calls to portmap aren't really needed. The
initial part of the script runs update and installs updates to the
distribution, then it installs nfs-server and copies the init.d file to
nfs as it is called by Starcluster scripts. Python is installed and
then SGE is downloaded, unzipped and renamed to /sge6-fresh so
Starcluster scripts work as advertised.
How to make your own Ubuntu 14.04 LTS AMI:
---------------------------------------
1) Find your AMI look here <
http://cloud-images.ubuntu.com/locator/ec2/>
for all Ubuntu 14.04 LTS AMIs for your region -- grab the AMI ID for my
example I used AMI ID: ami-36f8b15e -- its us-east-1, 14.04LTS, 64bit,
ebs-ssd
-- make sure to select and EBS backed EC2 AMI (not instance-store)
2) a) Login to AWS Console -- click on "launch an instance"
b) select "Community AMIs" and put the AMI id from step one in to
find it (here I used "ami-36f8b15e"), once it is found
click "Select"
c) Select instance type you want (choose something small because we
are just creating an AMI to use) - click NEXT
d) For "configure instance details" section -- make sure the
instance is placed in your default VPC -- just so you CAN access it if
need be
e) Under the "Advanced Details" section click the triangle/arrow so
that it expands. Paste in the following in the "User data" section. It
runs the bash script that will install needed packages for
making Starcluster work along with fixing issues with NFS and portmapper.
This ASSUMES you are not using the StarCluster master node as
StarCluster "master" AND StarCluster "controller". If you want to do
this add "easy_install StarCluster" (no quotes included) after
the last line "chmod +x /etc/init.d/portmap". Adding this line will
install StarCluster on the EC2 you are launching. After you
have pasted in the script in "User data" - click NEXT
#!/bin/bash
apt-get update
apt-get dist-upgrade -y
apt-get install nfs-kernel-server nfs-common rpcbind libgssglue1
libnfsidmap2 libtirpc1 -y
cp /etc/init.d/nfs-kernel-server /etc/init.d/nfs
apt-get install sysv-rc-conf -y
sysv-rc-conf nfs on
/etc/init.d/nfs restart
apt-get install python-setuptools python-dev -y
cd /opt/
wget
http://dl.dropbox.com/u/47200624/respin/ge2011.11.tar.gz
gunzip < ge2011.11.tar.gz | tar xvf -
mv ge2011.11/ sge6-fresh/
cd /
echo 'exit 0' > /etc/init.d/portmap
chmod +x /etc/init.d/portmap
f) Click Next - so that you are in the "Tag Instance" section and
Name your instance. I used "Ubuntu 14.04 Starcluster ready"
g) Click Next - so you are in "Configure Security Group" section --
here either select a pre-existing Security Group that you can
SSH into -- OR -- Create a new security group that allows you
to SSH into it from "My IP" (which is your IP).
h) Click Review and Launch
i) Click Launch
j) Select an existing Keypair you have access too (so you can SSH
into the instance) and click the check box and click "Launch
Instances" OR create a new keypair, download that keypair and
then click "Launch Instances".
3) Wait about 5-10 minutes and "Status Checks" should change from
"Initializing" to " 2/2 checks passed" -- You can login to the instance
and make sure issuing the command
"sudo vi /etc/init.d/portmap" opens the file and shows the command
"exit 0". This means the script completed and you can logout.
4) Stop the running instance by selecting "Actions - Instance State - Stop"
5) Select "Actions - Image - Create Image" to create the AMI -- name it
something like "Ubuntu 14.04 Starcluster node AMI" and then select
"Create Image" You will see "Create Image request received. View
pending image ami-XXXX" note that AMI ID -- and that's the AMI ID you
will use to create your Ubuntu 14.04 Starcluster master and worker nodes.
6) Once the AMI ID ami-XXXX - has Status "available" you can terminate
the EC2 Instance we launched with the User data script because it is no
longer needed. Use that AMI ID ami-XXXX in
the StarCluster config file AND/OR when you want to launch master
and worker nodes that are Ubuntu 14.04 LTS.
Potential Issues:
---------------
Step 1: DON'T choose "instance store" -- this isn't an EBS backed EC2 so
when it is stopped all adjustments to OS are lost. Use something that
says "ebs" in the instance type.
ebs = magnetic disk, ebs-ssd = the general purpose ssd storage,
ebs-ios = the Provisioned IOPS ssd storage.
Step 3: WAIT at least 5-10 minutes before stopping the EC2 Instance --
also it is really good idea to login to the EC2 and check that
/etc/init.d/portmap says "exit 0" so you
know that the script ran.
Virtualization type issue -- Specifically your AMI is PV or HVM and you
need the other type.
1) Go to Snapshots -- locate snapshot of AMI ID ami-XXX - note the
"Description" has AMI ID in its text. Select that snapshot.
2) Select "Actions - Create Image" -- note in "Virtualization type"
section you can select "Paravirtual" or "Hardware-assisted
virtualization" -- so select virtualization type you
need, Name the AMI, and select the other features you want then
click "Create". This creates the AMI of the opposite virtualization type
without any hassle. Using the
snapshot to create and AMI of the virualization of your choice
is WAY EASIER than the method I had used previously.
Good Luck!
-Jennifer
On 2/16/15 12:27 PM, Cedar McKay wrote:
> Some people, like Jennifer, have managed to get 14.04 AMIs working. It would be a real service if those people could share them.
>
> Also, does anyone have a reasonable explanation for why the official version is so slow in coming? Is there some sort of issue?
>
> Thanks,
> Cedar
>
>
>
>> On Jan 29, 2015, at 6:41 AM, Pablo Escobar<pescobar001_at_gmail.com> wrote:
>>
>> Hi Jennifer
>>
>> thank you very much for your help. It's really useful.
>>
>> regards,
>> Pablo.
>>
>> 2015-01-28 19:10 GMT+01:00 Jennifer Staab<jstaab_at_cs.unc.edu>:
>> I installed Starcluster on Ubuntu 14.04 LTS and it is working fine for me. I didn't use the guide you mentioned, just a little guidance from emails on mailing list between Jacob and Hugh regarding installation (see here) and some sleuthing to get it to work. Hugh also provides a guideline/cookbook for creating new Starcluster AMI's here on the mailing list and said that just upgrading the existing AMI for 12.04 to 14.04 worked for him. To me that seems like the easiest way to handle this whole issue is to use existing Ubuntu 12.04 Starcluster AMI and just upgrade it to 14.04. I wasn't able to do this, so below is what worked for me. I am sure there are other ways to create a new Ubuntu 14.04 Starcluster AMI.
>> --------------------------------------------------------------------------------------
>> 1) Locate the AMI ID of the latest version of Ubuntu 14.04 for your region (see here). Start up your Ubuntu 14.04 AMI -- login and run update and apply all the security and distribution updates.
>>
>> 2) Install packages "python-dev" and "python-setuptools" -- You are welcome to install other packages, but those are the only ones needed if to do the Starcluster install. A list of all python packages can be found here .
>>
>> 3) You need to install NFS server so install package "nfs-kernel-server". Installing this package should install its dependencies (for me that included: nfs-common rpcbind libgssglue1 libnfsidmap2 libtirpc1). One issue I found was that in previous versions the startup script located in /etc/init.d/ was named "nfs" this installed version it's named "nfs-kernel-server". The easiest fix is to create a symbolic link to "/etc/init.d/nfs" (ln -s /etc/init.d/nfs-kernel-server /etc/init.d/nfs). Test by issuing the command "/etc/init.d/nfs restart" if it restarts then all is well.
>>
>> 4) Install Starcluster with "easy_install Starcluster"
>>
>> 5) You need to get a copy of the Open Grid Scheduler and place it in "/opt/sge6-fresh/" -- this is what Hugh and Jacob discuss in their emails on the mailing list. The easiest way I was able to do this was to use a running Ubuntu 12.04 Starcluster AMI (ami-765b3e1f) and copy "/opt/sge-fresh" from the 12.04 version to my Ubuntu 14.04 version. There are other ways to do this see their emails here.
>>
>> 6) Ubuntu 14.04 replaces portmap with rpcbind -- so this was an issue for me because the Starcluster software expects "portmap" to exist. So this is where I had to "edit" the code a bit.There are other ways to deal with the portmap issue, look here for an alternative solution. This is just how I handled the issue: locate the "node.py" portion of the code in the installed StarCluster version, mine was in "/usr/local/lib/python2.7/dist-packages/StarclusterXXX.egg/starcluster/node.py" On line 731 in the function "mount_nfs_shares()" you see line "self.ssh.execute('/etc/init.d/portmap start')". This won't work on Ubuntu 14.04 since "portmap" was replaced with "rpcbind". One way to fix this issue without breaking the code for other OS versions still using portmap is by replacing "self.ssh.execute('/etc/init.d/portmap start')" with a "try/except" statement so that if portmap doesn't work it calls rpcbind command instead; thus, handling the issue with portmap on Ubuntu 14.04. B!
> elow is how I changed the code in node.py:
>> FIND line 731:
>> self.ssh.execute('/etc/init.d/portmap start')
>> REPLACE with:
>> try:
>> self.ssh.execute('/etc/init.d/portmap start')
>> except:
>> self.ssh.execute('service rpcbind restart')
>>
>> 7) Stop this instance and take an AMI of it. Use that AMI for nodes on your Starcluster clusters. If it bothers you to have Starcluster installed on the AMI you are using in your Starcluster clusters skip steps 2 and 4 BUT be certain to edit the version of Starcluster software you are using to create your Ubuntu 14.04 clusters to deal with portmap/rpcbind issue as described in step 6.
>>
>> A few things to note:
>>
>> 1) I only use starcluster commands to create clusters, terminate them, addnodes and removenodes (on-demand & spots) -- and this is what has been working for me. I haven't tested the other functionality, so there could be other issues like the portmap one that I don't know about. For the other functionality (like stopping instances, saving AMIs, etc), I use AWS directly to perform those functions.
>> 2) The version of StarCluster that I used was 0.95.6 -- as available via easy_install. These issues (or ones similar to them) have been discussed and a few fixed in some of the development branches of Starcluster available on github see here and here.
>> 3) I had some issue installing other software, but I am guessing most of that has been resolved now that it has been so long since the initial release of Ubuntu 14.04.
>>
>> Good Luck,
>> Jennifer
>>
>>
>> On 1/27/15 3:59 PM, Pablo Escobar wrote:
>>> Hi all,
>>>
>>> I plan to use starcluster with ubuntu14.04. I will adapt the ami following this guide
>>> https://www.youtube.com/watch?v=2RBupgpi_ec
>>>
>>> Is anyone in the list using starcluster with ubuntu 14.04 in production? Is it working fine or is there any drawback? is there any problem I should be aware before start using starcluster with ubuntu14.04?
>>>
>>> I know this has been asked before, but is there any plan to release a ubuntu14.04 official ami?
>>>
>>> thanks in advance for any advice.
>>>
>>> best regards,
>>> Pablo.
>>>
>>>
>>> _______________________________________________
>>> StarCluster mailing list
>>>
>>> StarCluster_at_mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Thu Feb 19 2015 - 15:38:17 EST