Re: adding spot instances without using starcluster commands

From: Rajat Banerjee <no email>
Date: Tue, 8 Mar 2016 10:46:07 -0500

No, it is a bad idea. You should look at the code for addnode and
introspect about why you would want to re-write that:

https://github.com/jtriley/StarCluster/blob/develop/starcluster/cluster.py#L998

Better to dig into why the spot instances are unable to communicate with
the master. Check their /etc/hosts file, dns connectivity, whether your
spot bid is too low and the instances are getting terminated. Remember
about spot bids -- no matter how high your bid, you'll get the lowest
available price. only when the price starts climbing do under-bidders get
affected.

On Tue, Mar 8, 2016 at 12:57 AM, rishi <rishiraaz_at_gmail.com> wrote:

> Hi Rajat,
> Thanks for prompt reply.
> addnode command is slow and spot instances added lose communication to sge
> master frequently ,not sure why.
>
> I suppose doing it without addnode will give more understanding and better
> control.
>
> Please guide/suggest a way to achieve this.
>
> On Mon, Mar 7, 2016 at 8:30 PM, Rajat Banerjee <rajatb_at_post.harvard.edu>
> wrote:
>
>> You will definitely want to use addnode, otherwise the new machines won't
>> be added to the oracle grid engine cluster, and you won't be able to give
>> them tasks or ssh to them.
>>
>> Have you looked at adding your scripts in the form of a plugin? It can
>> run within the context of addnode.
>>
>> There are ways to always use spot instances:
>> http://star.mit.edu/cluster/docs/latest/manual/configuration.html
>>
>> From the manual:
>>
>> The addnode command has additional options for customizing the new
>> node’s instance type, AMI, spot bid, and more. See the help menu for a
>> detailed list of all available options:
>>
>> $ starcluster addnode --help
>>
>> http://star.mit.edu/cluster/docs/latest/manual/addremovenode.html
>>
>>
>> On Mon, Mar 7, 2016 at 2:55 AM, rishi <rishiraaz_at_gmail.com> wrote:
>>
>>> Hi Team,
>>> I was wondering if there is a way to add spot instances to cluster
>>> without using starcluster commands , something like manual addition.
>>> The instances to be added are launched using a script and i want to be
>>> added to cluster tried a few things but without starcluser addnode command
>>> it is not working.
>>>
>>> Please suggest.
>>>
>>> --
>>> Rishi Raj Singh
>>> Cloud ;)
>>>
>>> _______________________________________________
>>> StarCluster mailing list
>>> StarCluster_at_mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>>
>>>
>>
>
>
> --
> Rishi Raj Singh
> Cloud ;)
>
Received on Tue Mar 08 2016 - 10:46:29 EST