Sergio,
It would be great if you can develep a web interface for StarCluster -
and I can help you test the code...
For queue status, Grid Engine (or "Open Grid Scheduler" - which is the
project that maintains the open source grid engine code base) has
XML-qstat, which was originally developed by Chris from the BioTeam,
and now it is available at:
http://olesenm.github.com/xml-qstat/
http://olesenm.github.com/xml-qstat/screenshots.html
There is also a web portal developed by Sun, but I don't think anyone
is using that one - at least not with any modern versions of Grid
Engine anyway.
Outside of Grid Engine, there is a lightweight web portal that handles
job submission, file upload, queue status, etc for OpenPBS (not Torque
- it was not tested with Torque, AFAIK), and the project is called
PBSWeb:
http://webdocs.cs.ualberta.ca/~pinchak/PBSWeb/
I talked to Paul Lu (the author) 11 years ago, and he said that PBSWeb
was developed by him & his grad students. He said that PBSWeb is
available under the GPL, and he was happy to accept changes from other
people. According to Paul, internally inside PBSWeb, the web front-end
invokes an SSH connection to the submission machine for each qsub, and
this was how he handled user switching (ie. the Apache user has to
have access to the secret keys of all the users - or else you would
see root or the Apache User as the submission user for all the jobs).
(BTW, Paul Lu was a student of Songnian Zhou, who was the founder of
Platform Computing, and Songnian wrote some of the initial Platform
LSF code when he was at UC Berkeley.)
However, as I am not a web programming guy, I didn't (and still don't)
know much about web technologies (Yes, I am using Gopher... just
kidding :-D ). Another thing to look at is the S3 multi-part upload,
GridFTP, and multi-session SCP upload methods:
http://www.elastician.com/2010/12/s3-multipart-upload-in-boto.html
http://www.globus.org/toolkit/docs/latest-stable/gridftp/
http://www.psc.edu/index.php/hpn-ssh
As you might know TCP is too slow to handle large file uploads, so
those technologies really help. I hope the web portal is flexible
enough to use at least those technologies by not hard coding to
specific way of doing the upload.
If you need changes in Grid Engine to integrate with the portal, then
I can help. Other than that, but then I am not a web guy and thus I am
not sure what's the best technology to use now in 2012 to tie these
things together - and may be in this area you know better than I do!
Rayson
================================
Open Grid Scheduler / Grid Engine
http://gridscheduler.sourceforge.net/
Scalable Grid Engine Support Program
http://www.scalablelogic.com/
On Wed, May 30, 2012 at 10:17 AM, Sergio Mafra <sergiohmafra_at_gmail.com> wrote:
> Hi fellows,
>
> One thing that is missing on a Cluster deployment is, in IMHO, a web site
> that could serve as a front side to the final users. This site should has
> some functionality like: Cluster Admin (on/off, cluster health),
> Upload/Download of files, queue status and others.
>
> I have a project to build one like this but IŽd like to know first if this
> is available in our community?
>
> All the best,
>
> Sergio
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
--
==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
Received on Wed May 30 2012 - 19:12:48 EDT