Hi Justin,
did you ever run into SGE problems after rebundling, i.e. commands
hanging, probably because of SSH troubles? I followed the normal
rebundling guidelines and the new instances come up but their logs contain:
/etc/init.d/rc: 390: /etc/rcS.d/S50ec2-ssh-host-key-gen: Permission denied
I googled a bit and it's down to Eric Hammond's original AMI (the one
the StarCluster one is based on) having this 'clever' script:
/etc/rcS.d/S50ec2-ssh-host-key-gen
which is set to make itself un-executable (chmod -x) after the very
first instantiation of an AMI. Could this be the cause for the SGE
commands not working anymore (hanging) after rebundling? I think it must
be since a bit of fiddling (sourcing the above script 'manually' and
then ./sge_master makes the commands not hang anymore but it also
doesn't fix things on the running instance).
Some people
(
http://developer.amazonwebservices.com/connect/message.jspa?messageID=150389)
suggest replacing the above script with this new version:
http://ec2ubuntu.googlecode.com/svn/trunk/etc/init.d/ec2-ssh-host-key-gen
or chmod +x the old script before rebundling
But before I go through the whole rebundling process again just to
replace this script I'd like to get your opinion on whether this causes
my problems? I'm afraid I myself know too little about this whole keys
business in conjunction with SGE.
Hope you can help,
Cheers,
Rob
starcluster-request_at_mit.edu wrote:
> Send Starcluster mailing list submissions to
> starcluster_at_mit.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mailman.mit.edu/mailman/listinfo/starcluster
> or, via email, send a message with subject or body 'help' to
> starcluster-request_at_mit.edu
>
> You can reach the person managing the list at
> starcluster-owner_at_mit.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Starcluster digest..."
>
>
> Today's Topics:
>
> 1. Re: WebFeedback - cluster: Hi again I guess it s better to
> use this than email Here s ... [help.mit.edu #1142117] (Justin Riley)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 1 Feb 2010 19:00:30 -0500
> From: Justin Riley <jtriley_at_MIT.EDU>
> Subject: Re: [Starcluster] WebFeedback - cluster: Hi again I guess it
> s better to use this than email Here s ... [help.mit.edu #1142117]
> To: rentzsch_at_biochem.ucl.ac.uk
> Cc: starcluster_at_mit.edu
> Message-ID: <201002011900.30417.jtriley_at_mit.edu>
> Content-Type: Text/Plain; charset="utf-8"
>
> Hi Rob,
>
> If you wouldn't mind, I'd prefer to use the starcluster
> (http://mailman.mit.edu/mailman/listinfo/starcluster) mailing list for
> reporting any issues/questions you might have in the future.
>
> Concerning your question, I'm guessing you mean is there a way to download the
> StarHPC AMI as a standalone virtual machine image? If so, I know that you can
> download the individual virtual machine files from S3 (ie the parts of the
> volume created by ec2-bundle-vol and the likes) but I've never tried to unpack
> them and get the vm running within a local Xen installation.
>
> This is not to say it couldn't be done, but I certainly don't have any
> experience with this. Regardless, the amount of money you'd save not storing a
> 10GB AMI on S3 you'd likely spend on the data transfer into S3 every time you
> wanted to launch the AMI. Furthermore, the process of transferring the image
> to s3, registering the AMI, and then launching the instance would make for an
> insanely long boot time. Honestly, I don't see the benefit.
>
> I typically do things the old fashioned way (launch instance, customize,
> rebundle) and also utilize EBS for housing research/application data and also
> applications themselves. Amazon allows you to create EBS volumes up to 1TB
> each. You can then mount these volumes to the instance. Starcluster supports
> this btw and nfs's the mounted EBS volume to all the nodes. Have a look at the
> docs if you're interested in using this.
>
> Interestingly enough, EC2 recently added support for booting from an EBS
> volume which opens up the amount of data that can be stored on an instance/AMI
> (up to 1TB). This feature is not available in the current stable version of
> StarCluster. Needless to say, I'll be experimenting with this in the very near
> future.
>
> Hope that helps,
>
> ~Justin
>
>
> On Monday 01 February 2010 12:26:56 pm rentzsch_at_biochem.ucl.ac.uk via RT
> wrote:
>
>> Mon Feb 01 12:26:55 2010: Request 1142117 was acted upon.
>> Transaction: Ticket created by rentzsch_at_biochem.ucl.ac.uk
>> Queue: StarIssues
>> Subject: WebFeedback - cluster: Hi again I guess it s better to use
>> this than email Here s ... Owner: Nobody
>> Requestors: rentzsch_at_biochem.ucl.ac.uk
>> Status: new
>> Ticket <URL: https://help.mit.edu/Ticket/Display.html?id=1142117 >
>>
>>
>>
>> Star WebFeedback
>> Time: Mon Feb 1 12:26:56 2010
>> Name: Rob
>> Project: cluster
>> Source: web
>> Report: Question
>> Email: rentzsch_at_biochem.ucl.ac.uk
>> Subscribe: on
>> Note: Hi again,
>>
>> I guess it's better to use this than email?
>>
>> Here's a more general question: is the unbundled StarCluster AMI available
>> from somewhere or will be? I might want to customize the AMI a bit
>> (software and maybe even some input data since I think an AMI can be 10
>> Gigs = using that could save S3 space?) and be able to to this 'offline'
>> i.e. mount it on my local machine.
>>
>> Don't worry if not, I'll try to do it the 'online' way then
>> (edit&rebundle).
>>
>> Cheers,
>> Rob
>>
>>
>
>
> ------------------------------
>
> _______________________________________________
> Starcluster mailing list
> Starcluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
> End of Starcluster Digest, Vol 6, Issue 2
> *****************************************
>
Received on Tue Feb 09 2010 - 06:48:33 EST