Are heterogeneous clusters possible in StarCluster?

From: Fran Campillo <no email>
Date: Mon, 08 Dec 2014 13:19:09 +0100

     Hi,

     I began using StarCluster a couple of weeks ago for my research,
and I find it really useful framework. I had to setup SGE myself several
times in the past, and StarCluster makes our life way easier.

     I still don't know many of the features of StarCluster, and I would
like to ask the community whether certain things I want to do are
actually possible with the current version from StarCluster. In
particular, I would like to create a heterogeneous cluster on
StarCluster (this is, with different kinds of instances, that I could
have in different SGE queues. In the problem I have to solve there are
some stages that need GPU and others that do not, and I would like to be
able to setup the complete cluster at the beginning of the process and
work like this:

------------

     1.- Init: setup of the heterogeneous cluster {gpu_clust, no_gpu_clust).

     2.- No cuda tasks:
     2.1.- stop gpu_clust instances.
     2.2.- run tasks in no_gpu_clust.

     3.- Cuda tasks:
     3.1.- stop no_gpu_clust.
     3.2.- start gpu_clust.
     3.3.- run tasks in gpu_clust.

     4.- No cuda tasks:
     ...

------------

     Is this currently possible with StarCluster? I guess that I can
already fake this behavior creating both clusters from another Amazon
instance and run the tasks with one or another via ssh, but I find it a
worse solution.

     Thank you very much in advance!

     Cheers!

     Fran.
Received on Mon Dec 08 2014 - 07:19:14 EST