Re: parallel ssh support?

From: Ryan G <no email>
Date: Tue, 21 Apr 2015 22:37:31 -0400

I figured out what's going on. When StarCluster creates a new cluster, it
reads the cluster template from the config file and knows the full path of
the userdata_script. This gets fed to amazon when creating new nodes.
This information is somehow encoded into a metatag (along with a whole
bunch of other information) on the security-group of the cluster.

When adding nodes to an existing cluster, the config file is never
referenced. The information about the cluster is encoded in a metatag on
the security group for the cluster. StarCluster reads the metadata of the
master node when creating additional nodes. In the metadata, the text (and
name) of the script is loaded into a userdata variable which never gets
propagated when creating new nodes.

If I feed just the body of script of the userdata of the master node, there
is no path specified so it would fail. I think the fullpath of the
userscript needs to be stored in the metadata of the securitygroup as the
other information about the cluster. I'll look into applying this and make
a patch.

If anyone more familiar with StarCluster knows more about this or can make
recommendations on a better fix, please let me know. Perhaps storing the
name of the cluster template then referencing the config file to determine
the user script but this wouldn't go along with how the rest of StarCluster
seems to work.

Ryan

Ryan

On Thu, Apr 16, 2015 at 5:32 PM, Ryan G <ngsbioinformatics_at_gmail.com> wrote:

> Yes, I've since discovered USERDATA_SCRIPTS doesn't work when adding
> nodes. Perhaps I'll take a look since this is a big issue for me.
>
>
> On Thu, Apr 16, 2015 at 1:42 PM, Cedar McKay <cmckay_at_uw.edu> wrote:
>
>> As an FYI, I’ve only been able to get USERDATA_SCRIPTS to work at cluster
>> launch. Addnode and loadbalance don’t seem to run the script, therefore the
>> feature is useless to me. I’d love to hear that I’m wrong.
>>
>>
>> > On Apr 16, 2015, at 10:37 AM, Ryan G <ngsbioinformatics_at_gmail.com>
>> wrote:
>> >
>> > Turns out this is supported by StarCluster in the config file using
>> USERDATA_SCRIPTS
>>
>>
>>
>
Received on Tue Apr 21 2015 - 22:37:33 EDT