I am not sure exactly what you are trying to do regarding a single
cluster. But in my experience the setup of the "master" node is
propagated to the worker nodes. I would set the master as I wanted and
then add "workers" that had different attributes ( different AMI's and
EC2 types, could be spots and/or on-demand). The workers always
inherited attributes from the master node, specifically NFS volumes from
the master were always shared amongst the workers ( this included other
EBS volumes I had mounted using Starcluster config file). When you
submit the "starcluster addnode" command you just have to make sure you
specify the attributes(AMI, instance type, spot/on-demand, etc.) you
want for the added nodes.
In my experience, the added worker nodes have all their resources
dedicated to the SGE queue(s). You would use "qconf" command to adjust
how you want things set up regarding each added node and how you want to
set up your queues. One good thing about this setup is you can have your
master as an on-demand or reserved instance and your workers as spot
instances (bid with cheaper hourly rates). This way jobs running on
spots that are terminated (due to bid pricing) are just resubmitted back
to the queue as long as you issue the resubmit option in your qsub calls.
One word of caution is that I didn't mixing and match on OS type or
virtualization type (PV, HVM) within a single cluster. My thought was
that there might be underlying incompatibilities in the systems such
that the propagation of attributes from master to worker nodes might not
work seamlessly.
Good Luck.
-Jennifer
On 12/9/14 11:30 AM, Fran Campillo wrote:
>
> Hi Jin!
>
> Thank you so much for your answer :) . Yes, I can try to do that,
> but then I would need to do manually some stuff in the new nodes that
> StarCluster usually takes care of (like the password-less ssh and
> sharing /home and potentially other EBS volumes). Is my assumption right?
>
> Thanks again!
>
> Fran.
>
> On 09.12.2014 16:52, Jin Yu wrote:
>> Hi Fran,
>>
>> You can start a master node first and then add different types of
>> nodes later. You may setup the SGE to define the job allocation
>> behavior among these nodes.
>>
>> -Jin
>>
>> On Mon, Dec 8, 2014 at 6:19 AM, Fran Campillo <fran_at_spitch.ch
>> <mailto:fran_at_spitch.ch>> wrote:
>>
>>
>> Hi,
>>
>> I began using StarCluster a couple of weeks ago for my research,
>> and I find it really useful framework. I had to setup SGE myself
>> several
>> times in the past, and StarCluster makes our life way easier.
>>
>> I still don't know many of the features of StarCluster, and
>> I would
>> like to ask the community whether certain things I want to do are
>> actually possible with the current version from StarCluster. In
>> particular, I would like to create a heterogeneous cluster on
>> StarCluster (this is, with different kinds of instances, that I could
>> have in different SGE queues. In the problem I have to solve
>> there are
>> some stages that need GPU and others that do not, and I would
>> like to be
>> able to setup the complete cluster at the beginning of the
>> process and
>> work like this:
>>
>> ------------
>>
>> 1.- Init: setup of the heterogeneous cluster {gpu_clust,
>> no_gpu_clust).
>>
>> 2.- No cuda tasks:
>> 2.1.- stop gpu_clust instances.
>> 2.2.- run tasks in no_gpu_clust.
>>
>> 3.- Cuda tasks:
>> 3.1.- stop no_gpu_clust.
>> 3.2.- start gpu_clust.
>> 3.3.- run tasks in gpu_clust.
>>
>> 4.- No cuda tasks:
>> ...
>>
>> ------------
>>
>> Is this currently possible with StarCluster? I guess that I can
>> already fake this behavior creating both clusters from another Amazon
>> instance and run the tasks with one or another via ssh, but I
>> find it a
>> worse solution.
>>
>> Thank you very much in advance!
>>
>> Cheers!
>>
>> Fran.
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu <mailto:StarCluster_at_mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
>>
>
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Tue Dec 09 2014 - 12:35:46 EST