Re: IPython parallel DirectView not distributing to starcluster nodes.

From: Austin So <no email>
Date: Sat, 4 Jul 2015 15:00:01 -0700

No error messages. I assumed it was that file since it was a config file, but what do I know except things get buggered up.

If you or anyone for that matter is actually interested in replicating, then these are the steps:

1. create 5-node cluster via starcluster using the HVM AMI.
2. login via IP[y] Notebook.
3. open and execute the previously attached notebook without setting the number of engines within IP[y]
        - determine the number of unique domain names being utilized
4. close/halt and set the number of engines in IP[y] to whatever you feel like
5. reopen and execute the previously attached notebook.
        - determine the number of unique domain names being utilized.

If you do not honestly see a difference in the domain name utilization, let me know. Or if you think that this is a spurious concern that doesn’t alter performance of the cluster, then I’d be interested in knowing because it is clear that by not assigning engines in IP[y], that I get utilization across all nodes.

A.



> On Jul 4, 2015, at 2:23 PM, MinRK <benjaminrk_at_gmail.com> wrote:
>
> Setting the engines in the notebook definitely does not rewrite the ipcluster_config file. What error do you see?
>
> On Sat, Jul 4, 2015 at 12:39 PM, Austin So <austin.so_at_tomabio.com <mailto:austin.so_at_tomabio.com>> wrote:
> Note to self: do not set engines on IP[y] notebook as this rewrites the ipcluster_config and apparently messes everything up.
>
> A.
>
Received on Sat Jul 04 2015 - 18:00:10 EDT