Meta-configuration options

From: David Johnston <no email>
Date: Tue, 27 May 2014 11:50:38 -0400

The config file is pretty good at allowing for generality. However, I am
thinking about how I would do it if I needed a script to change these
parameters.

I see a few ways of doing this. Maybe there are more. I am wondering if
there is a commended way and I am sure it may depend.

1) Some things can be done easily enough by manually changing a few things
by hand. For example, add another cluster type to config file once and then
a script can pick the right one automatically. Certainly works fine for
some things.

2) I believe the star cluster shell script allows for some over-rides at
command line.

3) You can write a plugin and run it by script after creation. This
probably gives you the most freedom. The only issue I see is that you lose
the nice feature that the config file can simply be copied somewhere and
used to document how the cluster was created and how it can be recreated in
the future. I guess this could be handled by writing the plugin to read a
config file of it's own and save this with the other. Using plugins seems
to require you to learn more of the underlying star cluster code.

4) Make config file templates that has variables in the file like
{{NUMBER_OF_NODES}} and then use something like jinja2 or mako template
engines to render a new config file from these via script. I started along
this route before I learned about the nice INCLUDE variable in [global] to
separate out sensitive things like credentials.

One benefit of template engines like mako, is that they give you a lot of
power. You can write conditionals and loops right into the template. Not
sure if there are real use cases for this but thought I would mention it.
It might be useful if you wanted to make a different configuration option
always turn on when you are in one region or have more than some number of
nodes etc.

There are other things like hacking the starcluster code but I'd rather not
do that because it makes upgrading difficult. That's what configuration
schemes are for.

Anyone want to comment about which they use the most.
Dave
Received on Tue May 27 2014 - 11:50:39 EDT