Re: Using MIT StarCluster on VPC

From: Sergio Mafra <no email>
Date: Thu, 13 Mar 2014 21:25:17 -0300

Hi Dmitry.

This plugin has been found on the internet and should be other option:
https://gist.github.com/adgaudio/9200030

all best,

Sérgio

Em quinta-feira, 13 de março de 2014, Sergio Mafra <sergiohmafra_at_gmail.com>
escreveu:

> Hi Dmitry and Rayson,
>
> Thanks for the explanation and tips for this situation that should be a
> common place when StarCluster moves to private subnets.
> Other option could be assigning a secondary IP to master using EC2 cli,
> where this IP is fixed.
>
> All best,
>
> Sérgio
>
> Em quinta-feira, 13 de março de 2014, Dmitry Serenbrennikov <
> dmitry_at_adchemy.com <javascript:_e(%7B%7D,'cvml','dmitry_at_adchemy.com');>>
> escreveu:
>
>> Hi Sergio,
>>
>> To add to Rayson's answer, perhaps you can consider using DNS entries
>> instead of fixed IP addresses?
>> In our case, that's what we are doing. (Honestly, I didn't even know you
>> could request specific IPs...)
>>
>> I wrote a small plugin to have starcluster automatically maintain Route53
>> entries for each master and the nodes.
>> Feel free to use it as is or modify:
>> https://gist.github.com/nitecoder/9536584#file-starcluster-route53-plugin
>>
>> By the way, this is only my second plugin ever, so I would very much
>> appreciate any critique / improvements from the list!
>>
>> Cheers!
>> -Dmitry
>>
>>
>> ________________________________________
>> From: starcluster-bounces_at_mit.edu <starcluster-bounces_at_mit.edu> on
>> behalf of Rayson Ho <raysonlogin_at_gmail.com>
>> Sent: Thursday, March 13, 2014 1:09 PM
>> To: Sergio Mafra
>> Cc: starcluster_at_mit.edu
>> Subject: Re: [StarCluster] Using MIT StarCluster on VPC
>>
>> On Thu, Mar 13, 2014 at 3:34 PM, Sergio Mafra <sergiohmafra_at_gmail.com>
>> wrote:
>> > Since we´d like to assign to each master, a specific private ip, I´d
>> like to
>> > know if this can be done in StarCluster or by CLI commands. so my users
>> can
>> > always connect in their machines using the same procedure.
>>
>> Short answer: currently there is no support for that in StarCluster.
>>
>> Longer answer: While the boto (AWS Python) run_instances() API does
>> offer support for assigning specific private IP address (by using
>> parameter "private_ip_address") in a VPC subnet, there is a limitation
>> of starting only 1 instance when private_ip_address is specified.
>>
>> Thus, if you want to do it that way, the StarCluster code will need to
>> pass in the "private_ip_address" parameter when starting the master
>> node. Then the slave nodes will need to be started in another
>> run_instances() call. It should be doable (with some code changes) as
>> just running a 1-node cluster, then adding new nodes is my recommended
>> way of starting large clusters in EC2.
>>
>> Rayson
>>
>> ==================================================
>> Open Grid Scheduler - The Official Open Source Grid Engine
>> http://gridscheduler.sourceforge.net/
>> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>>
>>
>> >
>> > All best,
>> >
>> > Sergio
>> >
>> >
>> > _______________________________________________
>> > StarCluster mailing list
>> > StarCluster_at_mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/starcluster
>> >
>>
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
Received on Thu Mar 13 2014 - 20:25:19 EDT