Hi Paolo/Joydeep,
You're correct that StarCluster currently uses the user-data to store
node-aliases which at the moment kills the ability to utilize user-data
scripts. It is possible to run user-scripts on the entire cluster fairly
easily, however, using a StarCluster plugin. I'm happy to provide an
example or perhaps include it in a future release if you're interested.
The reason for storing the alias in user-data is that I needed some way
to 'stick' a name to a node at launch time. The new tagging API for EC2
is useful but it's still not possible to assign names at launch time. My
current approach is to use the user-data at launch time, then later
fetch it from user-data once the instance is up and apply an alias tag.
This is useful for example if things crash somewhere in the middle - the
nodes are guaranteed to be named the same thing the next time around
because of the user-data.
Your solution of effectively 'commenting' the alias line in user-data so
that other scripts can run is not a bad solution actually. I'd be
willing to accept a pull request for this and include the functionality
in a future release. We need to add a USER_DATA config parameter to the
cluster config that would let users point to a user-data script.
Would you be interested in contributing this back? I'm happy to help you
along with the fork/pull request process if you're willing. There's also
a contribute guide that gives an overview of the process:
http://web.mit.edu/star/cluster/docs/latest/contribute.html
~Justin
On 01/24/2012 11:41 AM, Joydeep Sen Sarma wrote:
> It doesn't.
>
> Currently the user data is used by starcluster itself to store node aliases.
>
> I had a similar requirement as yours - but in my case - the user data
> was restricted to shell scripts. So what i did for my use case was hack
> the starcluster code to use the first line of the user data - prepended
> with a '#' for node aliases - and the rest of the user data could be
> used as a shell script (filled in when launching the cluster from the
> configuration).
>
> it seemed pretty hacky to me - so didn't try to contribute back.
>
> there may be other ways of achieving this outcome. for example - one
> could take the cluster-name and the instance alias (node001 etc.) and
> then map that to a location in s3 (s3://urbucket/cluster-name/node001)
> and then store instance specific user data in that location. the boot
> scripts can get the cluster name and the instance alias from ec2 apis
> and then use that to locate the instance data and get rolling.
>
> On Tue, Jan 24, 2012 at 9:22 PM, Paolo Di Tommaso
> <Paolo.DiTommaso_at_crg.eu <mailto:Paolo.DiTommaso_at_crg.eu>> wrote:
>
> Dear all,
>
> Does StarCluster provide any support for EC2 user data feature?
>
> http://www.turnkeylinux.org/blog/ec2-userdata
>
>
> I would like to use this mechanism to specify some extra instance
> configuration at node boot time.
>
> If no, is there any other alternative?
>
>
> Thank you,
> Paolo Di Tommaso
>
> Software Engineer
> Comparative Bioinformatics Group
> Centre de Regulacio Genomica (CRG)
> Dr. Aiguader, 88
> 08003 Barcelona, Spain
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu <mailto:StarCluster_at_mit.edu>
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Tue Feb 07 2012 - 12:33:04 EST