Re: sge submit host question

From: Robert Yu <no email>
Date: Sat, 25 Feb 2012 03:02:07 -0800

Thanks.

For now, I'll go with the simple sshmaster command method.

I created a file ~sgeadmin/.sshrc to provide the same environment as
an interactive session. It looks like:

#------------------------------------------------------------------
# Source this file manually when running starcluster commands.
    # e.g. % starcluster sshmaster mycluster "source ~/.sshrc; command "
# Set this to avoid exiting out early:
export PS1=starcluster
source /etc/profile
# This in turn sources ~/.bashrc:
source ~/.profile
#------------------------------------------------------------------

-Robert

On Fri, Feb 24, 2012 at 8:56 PM, Rayson Ho <raysonlogin_at_yahoo.com> wrote:
> Robert,
>
> I spent some time on this this evening, and basically if you follow the steps I
> listed, PLUS if you set up the security group to allow traffic to port 63231
> (should be very easy if you use the Management Console), then you can have your
> local machine submitting jobs to the cluster on EC2.
>
> *However*, I recommend against this, because SGE traffic is not encrypted.
>
> So, the best way is to use the sshmaster command to remotely execute qsub.
>
> You might find that it is not as convenient, as for example you will need to
> setup environment variables, etc on the EC2 machines. (I also did try SSH port
> forwarding but the SGE protocol will simply block the connection as the qmaster
> finds it weird that the source IP comes from the host on EC2, while qsub claims
> that it is from your local machine.)
>
> If you have a use case that would strongly benefit from SSH port forwarding,
> please let me know.
>
>
> Rayson
>
> =================================
> Open Grid Scheduler / Grid Engine
> http://gridscheduler.sourceforge.net/
>
> Scalable Grid Engine Support Program
> http://www.scalablelogic.com/
>
>
>
>
>
>
> ----- Original Message -----
> From: Justin Riley <jtriley_at_mit.edu>
> To: David Erickson <derickso_at_stanford.edu>
> Cc: Robert Yu <robert.yu_at_aditazz.com>; starcluster <starcluster_at_mit.edu>
> Sent: Friday, February 24, 2012 4:36 PM
> Subject: Re: [StarCluster] sge submit host question
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Yes the environment issue is related to the fact that SSH does not
> source /etc/profile by default. For the time being you can get around
> this using:
>
> $ starcluster sshmaster mycluster 'source /etc/profile; <command>'
>
> Sourcing /etc/profile should probably be done by default when running
> commands using sshmaster/sshnode. I created an issue to keep track of
> this:
>
> http://web.mit.edu/star/cluster/issues/84
>
> HTH,
>
> ~Justin
>
>
> On 02/24/2012 03:38 PM, David Erickson wrote:
>> On 2/24/2012 12:07 PM, Robert Yu wrote:
>>> Hi,
>>>
>>> This is more of a best-practice question.
>>>
>>> Let's say I am on machine A.  From it, I create a cluster.  I can
>>> then ssh into the "master" and launch jobs.  Can this also be
>>> done from machine A directly?  I know one can do "starcluster
>>> sshmaster -u sgeadmin<clustername>  'some command'" but then I
>>> typically run into environment issues.  Is there (simple) a way
>>> to make machine A an sge submit host?
>>>
>>
>> I ran into this too and found out I had to source
>> /etc/profile.d/sge.sh.  I also am curious if there is a better
>> solution, presumably this is an interactive vs noninteractive shell
>> issue.
>>
>> -D _______________________________________________ StarCluster
>> mailing list StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk9IAsIACgkQ4llAkMfDcrmLdQCbBH1KCzDASMcUx0smaFSWU9Gx
> HY4An0QKuzBCpEi7ZCuQ06bY9LcRcPzw
> =T/Lq
> -----END PGP SIGNATURE-----
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>

-- 
Robert Yu, Member Technical Staff
www.aditazz.com | robert.yu_at_aditazz.com
1111 Bayhill Drive Suite 260 | San Bruno | CA 94066
510.459.0216 | cell
650.627.7357 | 650.492.7000 x1008 | work
650.684.1149 | fax

Received on Sat Feb 25 2012 - 06:02:38 EST