Re: Are heterogeneous clusters possible in StarCluster?

From: Fran Campillo <no email>
Date: Tue, 09 Dec 2014 17:30:20 +0100

     Hi Jin!

     Thank you so much for your answer :) . Yes, I can try to do that,
but then I would need to do manually some stuff in the new nodes that
StarCluster usually takes care of (like the password-less ssh and
sharing /home and potentially other EBS volumes). Is my assumption right?

     Thanks again!

     Fran.

On 09.12.2014 16:52, Jin Yu wrote:
> Hi Fran,
>
> You can start a master node first and then add different types of
> nodes later. You may setup the SGE to define the job allocation
> behavior among these nodes.
>
> -Jin
>
> On Mon, Dec 8, 2014 at 6:19 AM, Fran Campillo <fran_at_spitch.ch
> <mailto:fran_at_spitch.ch>> wrote:
>
>
> Hi,
>
> I began using StarCluster a couple of weeks ago for my research,
> and I find it really useful framework. I had to setup SGE myself
> several
> times in the past, and StarCluster makes our life way easier.
>
> I still don't know many of the features of StarCluster, and I
> would
> like to ask the community whether certain things I want to do are
> actually possible with the current version from StarCluster. In
> particular, I would like to create a heterogeneous cluster on
> StarCluster (this is, with different kinds of instances, that I could
> have in different SGE queues. In the problem I have to solve there are
> some stages that need GPU and others that do not, and I would like
> to be
> able to setup the complete cluster at the beginning of the process and
> work like this:
>
> ------------
>
> 1.- Init: setup of the heterogeneous cluster {gpu_clust,
> no_gpu_clust).
>
> 2.- No cuda tasks:
> 2.1.- stop gpu_clust instances.
> 2.2.- run tasks in no_gpu_clust.
>
> 3.- Cuda tasks:
> 3.1.- stop no_gpu_clust.
> 3.2.- start gpu_clust.
> 3.3.- run tasks in gpu_clust.
>
> 4.- No cuda tasks:
> ...
>
> ------------
>
> Is this currently possible with StarCluster? I guess that I can
> already fake this behavior creating both clusters from another Amazon
> instance and run the tasks with one or another via ssh, but I find
> it a
> worse solution.
>
> Thank you very much in advance!
>
> Cheers!
>
> Fran.
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu <mailto:StarCluster_at_mit.edu>
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
Received on Tue Dec 09 2014 - 11:30:34 EST