Robert,
I spent some time on this this evening, and basically if you follow the steps I
listed, PLUS if you set up the security group to allow traffic to port 63231
(should be very easy if you use the Management Console), then you can have your
local machine submitting jobs to the cluster on EC2.
*However*, I recommend against this, because SGE traffic is not encrypted.
So, the best way is to use the sshmaster command to remotely execute qsub.
You might find that it is not as convenient, as for example you will need to
setup environment variables, etc on the EC2 machines. (I also did try SSH port
forwarding but the SGE protocol will simply block the connection as the qmaster
finds it weird that the source IP comes from the host on EC2, while qsub claims
that it is from your local machine.)
If you have a use case that would strongly benefit from SSH port forwarding,
please let me know.
Rayson
=================================
Open Grid Scheduler / Grid Engine
http://gridscheduler.sourceforge.net/
Scalable Grid Engine Support Program
http://www.scalablelogic.com/
----- Original Message -----
From: Justin Riley <jtriley_at_mit.edu>
To: David Erickson <derickso_at_stanford.edu>
Cc: Robert Yu <robert.yu_at_aditazz.com>; starcluster <starcluster_at_mit.edu>
Sent: Friday, February 24, 2012 4:36 PM
Subject: Re: [StarCluster] sge submit host question
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Yes the environment issue is related to the fact that SSH does not
source /etc/profile by default. For the time being you can get around
this using:
$ starcluster sshmaster mycluster 'source /etc/profile; <command>'
Sourcing /etc/profile should probably be done by default when running
commands using sshmaster/sshnode. I created an issue to keep track of
this:
http://web.mit.edu/star/cluster/issues/84
HTH,
~Justin
On 02/24/2012 03:38 PM, David Erickson wrote:
> On 2/24/2012 12:07 PM, Robert Yu wrote:
>> Hi,
>>
>> This is more of a best-practice question.
>>
>> Let's say I am on machine A. From it, I create a cluster. I can
>> then ssh into the "master" and launch jobs. Can this also be
>> done from machine A directly? I know one can do "starcluster
>> sshmaster -u sgeadmin<clustername> 'some command'" but then I
>> typically run into environment issues. Is there (simple) a way
>> to make machine A an sge submit host?
>>
>
> I ran into this too and found out I had to source
> /etc/profile.d/sge.sh. I also am curious if there is a better
> solution, presumably this is an interactive vs noninteractive shell
> issue.
>
> -D _______________________________________________ StarCluster
> mailing list StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla -
http://enigmail.mozdev.org/
iEYEARECAAYFAk9IAsIACgkQ4llAkMfDcrmLdQCbBH1KCzDASMcUx0smaFSWU9Gx
HY4An0QKuzBCpEi7ZCuQ06bY9LcRcPzw
=T/Lq
-----END PGP SIGNATURE-----
_______________________________________________
StarCluster mailing list
StarCluster_at_mit.edu
http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Fri Feb 24 2012 - 23:56:50 EST