Re: sge submit host question

From: Justin Riley <no email>
Date: Fri, 24 Feb 2012 16:47:24 -0500

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Chris,

By default all of the nodes in the cluster *should* be admin, submit,
and exec hosts.

With that said I'd love for folks to start customizing the new SGE
plugin on git (will be released with 0.93.2 as well). You can find the
new plugin in "plugins/sge.py".

Each SGE customization should be implemented as an optional keyword
argument to the plugin's __init__ method - other plugin methods should
then read these parameters and perform the customizations in run(),
on_add_node(), etc. I'm happy to help folks get started if interested.

~Justin

On 02/24/2012 03:58 PM, Chris Dagdigian wrote:
>
> It is *extremely* common for all SGE nodes in a cluster to be both
> admin and submit hosts.
>
> The reason for making every node a submit host is that there are
> cluster workflows that often include jobs running on a node that
> either do new job submissions directly or else alter things like
> hold_jid or the parameters of a job array.
>
> For small clusters that are persistant, the CLI command is "qconf
> -as <hostname>" to add a submit host.
>
> Starcluster uses an SGE autoinstallation template file (I think
> ...) to automate the process of cluster building. Within this
> template file there is a parameter where you can list out your
> Admin and Submit hosts.
>
> I imagine it would be pretty simple to customize or override the
> default StarCluster autoinstallation file so that all your nodes
> become submit host.
>
> And if that is too complicated it's pretty trivial to iterate over
> the hostnames and run the "qconf -as" command ...
>
> My $.02 of course!
>
> -Chris
>
>
>> David Erickson <mailto:derickso_at_stanford.edu> February 24, 2012
>> 3:38 PM
>>
>> I ran into this too and found out I had to source
>> /etc/profile.d/sge.sh. I also am curious if there is a better
>> solution, presumably this is an interactive vs noninteractive
>> shell issue.
>>
>> -D _______________________________________________ StarCluster
>> mailing list StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster Robert Yu
>> <mailto:robert.yu_at_aditazz.com> February 24, 2012 3:07 PM Hi,
>>
>> This is more of a best-practice question.
>>
>> Let's say I am on machine A. From it, I create a cluster. I can
>> then ssh into the "master" and launch jobs. Can this also be done
>> from machine A directly? I know one can do "starcluster sshmaster
>> -u sgeadmin <clustername> 'some command'" but then I typically
>> run into environment issues. Is there (simple) a way to make
>> machine A an sge submit host?
>>
>
> *Chris Dagdigian*/Principal Consultant//, BioTeam Inc./
> http://bioteam.net SlideShare
> <http://www.slideshare.net/chrisdag/> Twitter
> <https://twitter.com/chris_dag> LinkedIn
> <http://www.linkedin.com/in/chrisdag> Flickr
> <http://flic.kr/s/aHsiPuSdtY>
>
> My latest BioTeam Blog Post: Metrum moves to the cloud
> <http://bioteam.net/2012/02/metrum-moves-to-the-cloud/>
>
>
> This body part will be downloaded on demand.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9IBWsACgkQ4llAkMfDcrlgLACfUYrrpRYk3hPu9SUgw6w+9hRe
KIoAn39PejiVJf861cfT+KEYzhhaMtzS
=3hpo
-----END PGP SIGNATURE-----
Received on Fri Feb 24 2012 - 16:47:27 EST