Re: One simple question

From: Austin Godber <no email>
Date: Sat, 23 Oct 2010 14:53:48 -0400

On 10/23/2010 02:47 PM, Justin Riley wrote:
> On 10/23/10 2:19 PM, Alexey PETROV wrote:
>> And, could I just confirm, once again - "If a single user need to run
>> a MPI task just from time to time (not on routine everyday basis),
>> would he have some additional benefits from "queuing system" in a
>> cloud, or it better to use MPI straightforward"?
> In the case of MPI jobs that use the entire cluster for a single job,
> the queuing system isn't as crucial, although it can still be
> beneficial. For example, if you want to run more than one MPI job task
> one after the other then queuing each of these jobs would allow you to
> submit them and forget about them.
>
> Also, OpenMPI is an SGE-aware MPI implementation. This means that if you
> qsub an 'mpirun' command, the OpenMPI runtime will recognize that it's
> being executed by SGE and fetch the list of hosts to use from SGE rather
> than having to build your own hostfile for the jobs. For more details on
> using Sun Grid Engine and MPI have a look at the this thread:
>
> http://mailman.mit.edu/pipermail/starcluster/2010-June/000264.html
>
> Hope that helps,
>
> ~Justin
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>

I also think it is worth noting that StarCluster doesn't just provide
the SGE queuing system. It builds the whole coherent environment. You
get your nodes, preconfigured with users (and ssh keys in place) and
shared EBS backed disk space. I haven't seen anything else that
provides all that as easily as StarCluster. You would have to reproduce
at least some of that to get your MPI environment.

Providing SGE (plus MPI) in EC2 may be the main goal of StarCluster, but
it provides a much more generic resource than that. In my opinion
theres a lot of untapped value in that.

Austin
Received on Sat Oct 23 2010 - 14:53:51 EDT