I installed Starcluster on Ubuntu 14.04 LTS and it is working fine for
me. I didn't use the guide you mentioned, just a little guidance from
emails on mailing list between Jacob and Hugh regarding installation (see
here <
http://star.mit.edu/cluster/mlarchives/2389.html>) and some sleuthing
to get it to work. Hugh also provides a guideline/cookbook for creating
new Starcluster AMI's here
<
http://star.mit.edu/cluster/mlarchives/2353.html> on the mailing list and
said that just upgrading the existing AMI for 12.04 to 14.04 worked for
him. To me that seems like the easiest way to handle this whole issue is to
use existing Ubuntu 12.04 Starcluster AMI and just upgrade it to 14.04. I
wasn't able to do this, so below is what worked for me. I am sure there are
other ways to create a new Ubuntu 14.04 Starcluster AMI.
--------------------------------------------------------------------------------------
1) Locate the AMI ID of the latest version of Ubuntu 14.04 for your region
(see here <
http://cloud-images.ubuntu.com/locator/ec2/>). Start up your
Ubuntu 14.04 AMI -- login and run update and apply all the security and
distribution updates.
2) Install packages "python-dev" and "python-setuptools" -- You are welcome
to install other packages, but those are the only ones needed if to do the
Starcluster install. A list of all python packages can be found here
<
http://packages.ubuntu.com/source/trusty/python/> .
3) You need to install NFS server so install package "nfs-kernel-server".
Installing this package should install its dependencies (for me that
included: nfs-common rpcbind libgssglue1 libnfsidmap2 libtirpc1). One issue
I found was that in previous versions the startup script located in
/etc/init.d/ was named "nfs" this installed version it's named
"nfs-kernel-server". The easiest fix is to create a symbolic link to
"/etc/init.d/nfs" (ln -s /etc/init.d/nfs-kernel-server /etc/init.d/nfs).
Test by issuing the command "/etc/init.d/nfs restart" if it restarts then
all is well.
4) Install Starcluster with "easy_install Starcluster"
5) You need to get a copy of the Open Grid Scheduler and place it in
"/opt/sge6-fresh/" -- this is what Hugh and Jacob discuss in their emails
on the mailing list. The easiest way I was able to do this was to use a
running Ubuntu 12.04 Starcluster AMI (ami-765b3e1f) and copy
"/opt/sge-fresh" from the 12.04 version to my Ubuntu 14.04 version. There
are other ways to do this see their emails here
<
http://star.mit.edu/cluster/mlarchives/2389.html>.
6) Ubuntu 14.04 replaces portmap with rpcbind -- so this was an issue for
me because the Starcluster software expects "portmap" to exist. So this is
where I had to "edit" the code a bit.There are other ways to deal with the
portmap issue, look here <
http://star.mit.edu/cluster/mlarchives/2353.html> for
an alternative solution. This is just how I handled the issue: locate the
"node.py" portion of the code in the installed StarCluster version, mine
was in
"/usr/local/lib/python2.7/dist-packages/StarclusterXXX.egg/starcluster/node.py"
On line 731 in the function "mount_nfs_shares()" you see line
"self.ssh.execute('/etc/init.d/portmap start')". This won't work on Ubuntu
14.04 since "portmap" was replaced with "rpcbind". One way to fix this
issue without breaking the code for other OS versions still using portmap
is by replacing "self.ssh.execute('/etc/init.d/portmap start')" with a
"try/except" statement so that if portmap doesn't work it calls rpcbind
command instead; thus, handling the issue with portmap on Ubuntu 14.04.
Below is how I changed the code in node.py:
FIND line 731:
self.ssh.execute('/etc/init.d/portmap start')
REPLACE with:
try:
self.ssh.execute('/etc/init.d/portmap start')
except:
self.ssh.execute('service rpcbind restart')
7) Stop this instance and take an AMI of it. Use that AMI for nodes on
your Starcluster clusters. If it bothers you to have Starcluster installed
on the AMI you are using in your Starcluster clusters skip steps 2 and 4
BUT be certain to edit the version of Starcluster software you are using to
create your Ubuntu 14.04 clusters to deal with portmap/rpcbind issue as
described in step 6.
A few things to note:
1) I only use starcluster commands to create clusters, terminate them,
addnodes and removenodes (on-demand & spots) -- and this is what has been
working for me. I haven't tested the other functionality, so there could be
other issues like the portmap one that I don't know about. For the other
functionality (like stopping instances, saving AMIs, etc), I use AWS
directly to perform those functions.
2) The version of StarCluster that I used was 0.95.6 -- as available via
easy_install. These issues (or ones similar to them) have been discussed
and a few fixed in some of the development branches of Starcluster
available on github see here
<
https://github.com/jtriley/StarCluster/issues?q=portmap> and here
<
https://github.com/jtriley/StarCluster/issues?q=ubuntu+14.04>.
3) I had some issue installing other software, but I am guessing most of
that has been resolved now that it has been so long since the initial
release of Ubuntu 14.04.
Good Luck,
Jennifer
On 1/27/15 3:59 PM, Pablo Escobar wrote:
Hi all,
I plan to use starcluster with ubuntu14.04. I will adapt the ami following
this guide
https://www.youtube.com/watch?v=2RBupgpi_ec
Is anyone in the list using starcluster with ubuntu 14.04 in production?
Is it working fine or is there any drawback? is there any problem I should
be aware before start using starcluster with ubuntu14.04?
I know this has been asked before, but is there any plan to release a
ubuntu14.04 official ami?
thanks in advance for any advice.
best regards,
Pablo.
_______________________________________________
StarCluster mailing
listStarCluster_at_mit.edu
http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Wed Jan 28 2015 - 13:10:36 EST