Re: parallel ssh support?

From: Ryan G <no email>
Date: Wed, 22 Apr 2015 15:23:34 -0400

I submitted a pull request that fixes this:

https://github.com/golharam/StarCluster/commit/e4b776f2d4fd18827cb77d8e5422dc5dadcc6e31


On Tue, Apr 21, 2015 at 10:37 PM, Ryan G <ngsbioinformatics_at_gmail.com>
wrote:

> I figured out what's going on. When StarCluster creates a new cluster, it
> reads the cluster template from the config file and knows the full path of
> the userdata_script. This gets fed to amazon when creating new nodes.
> This information is somehow encoded into a metatag (along with a whole
> bunch of other information) on the security-group of the cluster.
>
> When adding nodes to an existing cluster, the config file is never
> referenced. The information about the cluster is encoded in a metatag on
> the security group for the cluster. StarCluster reads the metadata of the
> master node when creating additional nodes. In the metadata, the text (and
> name) of the script is loaded into a userdata variable which never gets
> propagated when creating new nodes.
>
> If I feed just the body of script of the userdata of the master node,
> there is no path specified so it would fail. I think the fullpath of the
> userscript needs to be stored in the metadata of the securitygroup as the
> other information about the cluster. I'll look into applying this and make
> a patch.
>
> If anyone more familiar with StarCluster knows more about this or can make
> recommendations on a better fix, please let me know. Perhaps storing the
> name of the cluster template then referencing the config file to determine
> the user script but this wouldn't go along with how the rest of StarCluster
> seems to work.
>
> Ryan
>
> Ryan
>
> On Thu, Apr 16, 2015 at 5:32 PM, Ryan G <ngsbioinformatics_at_gmail.com>
> wrote:
>
>> Yes, I've since discovered USERDATA_SCRIPTS doesn't work when adding
>> nodes. Perhaps I'll take a look since this is a big issue for me.
>>
>>
>> On Thu, Apr 16, 2015 at 1:42 PM, Cedar McKay <cmckay_at_uw.edu> wrote:
>>
>>> As an FYI, I’ve only been able to get USERDATA_SCRIPTS to work at
>>> cluster launch. Addnode and loadbalance don’t seem to run the script,
>>> therefore the feature is useless to me. I’d love to hear that I’m wrong.
>>>
>>>
>>> > On Apr 16, 2015, at 10:37 AM, Ryan G <ngsbioinformatics_at_gmail.com>
>>> wrote:
>>> >
>>> > Turns out this is supported by StarCluster in the config file using
>>> USERDATA_SCRIPTS
>>>
>>>
>>>
>>
>
Received on Wed Apr 22 2015 - 15:23:38 EDT