Re: Using MIT StarCluster on VPC

From: Sergio Mafra <no email>
Date: Thu, 13 Mar 2014 21:09:36 -0300

Hi Dmitry and Rayson,

Thanks for the explanation and tips for this situation that should be a
common place when StarCluster moves to private subnets.
Other option could be assigning a secondary IP to master using EC2 cli,
where this IP is fixed.

All best,

Sérgio

Em quinta-feira, 13 de março de 2014, Dmitry Serenbrennikov <
dmitry_at_adchemy.com> escreveu:

> Hi Sergio,
>
> To add to Rayson's answer, perhaps you can consider using DNS entries
> instead of fixed IP addresses?
> In our case, that's what we are doing. (Honestly, I didn't even know you
> could request specific IPs...)
>
> I wrote a small plugin to have starcluster automatically maintain Route53
> entries for each master and the nodes.
> Feel free to use it as is or modify:
> https://gist.github.com/nitecoder/9536584#file-starcluster-route53-plugin
>
> By the way, this is only my second plugin ever, so I would very much
> appreciate any critique / improvements from the list!
>
> Cheers!
> -Dmitry
>
>
> ________________________________________
> From: starcluster-bounces_at_mit.edu <javascript:;> <
> starcluster-bounces_at_mit.edu <javascript:;>> on behalf of Rayson Ho <
> raysonlogin_at_gmail.com <javascript:;>>
> Sent: Thursday, March 13, 2014 1:09 PM
> To: Sergio Mafra
> Cc: starcluster_at_mit.edu <javascript:;>
> Subject: Re: [StarCluster] Using MIT StarCluster on VPC
>
> On Thu, Mar 13, 2014 at 3:34 PM, Sergio Mafra <sergiohmafra_at_gmail.com<javascript:;>>
> wrote:
> > Since we´d like to assign to each master, a specific private ip, I´d
> like to
> > know if this can be done in StarCluster or by CLI commands. so my users
> can
> > always connect in their machines using the same procedure.
>
> Short answer: currently there is no support for that in StarCluster.
>
> Longer answer: While the boto (AWS Python) run_instances() API does
> offer support for assigning specific private IP address (by using
> parameter "private_ip_address") in a VPC subnet, there is a limitation
> of starting only 1 instance when private_ip_address is specified.
>
> Thus, if you want to do it that way, the StarCluster code will need to
> pass in the "private_ip_address" parameter when starting the master
> node. Then the slave nodes will need to be started in another
> run_instances() call. It should be doable (with some code changes) as
> just running a 1-node cluster, then adding new nodes is my recommended
> way of starting large clusters in EC2.
>
> Rayson
>
> ==================================================
> Open Grid Scheduler - The Official Open Source Grid Engine
> http://gridscheduler.sourceforge.net/
> http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html
>
>
> >
> > All best,
> >
> > Sergio
> >
> >
> > _______________________________________________
> > StarCluster mailing list
> > StarCluster_at_mit.edu <javascript:;>
> > http://mailman.mit.edu/mailman/listinfo/starcluster
> >
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu <javascript:;>
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Thu Mar 13 2014 - 20:09:38 EDT