StarCluster - Mailing List Archive

Re: IPython parallel DirectView not distributing to starcluster nodes.

From: Austin So <no email>
Date: Sat, 4 Jul 2015 15:00:01 -0700

No error messages. I assumed it was that file since it was a config file, but what do I know except things get buggered up.

If you or anyone for that matter is actually interested in replicating, then these are the steps:

1. create 5-node cluster via starcluster using the HVM AMI.
2. login via IP[y] Notebook.
3. open and execute the previously attached notebook without setting the number of engines within IP[y]
        - determine the number of unique domain names being utilized
4. close/halt and set the number of engines in IP[y] to whatever you feel like
5. reopen and execute the previously attached notebook.
        - determine the number of unique domain names being utilized.

If you do not honestly see a difference in the domain name utilization, let me know. Or if you think that this is a spurious concern that doesn’t alter performance of the cluster, then I’d be interested in knowing because it is clear that by not assigning engines in IP[y], that I get utilization across all nodes.


> On Jul 4, 2015, at 2:23 PM, MinRK <> wrote:
> Setting the engines in the notebook definitely does not rewrite the ipcluster_config file. What error do you see?
> On Sat, Jul 4, 2015 at 12:39 PM, Austin So < <>> wrote:
> Note to self: do not set engines on IP[y] notebook as this rewrites the ipcluster_config and apparently messes everything up.
> A.
Received on Sat Jul 04 2015 - 18:00:10 EDT
This archive was generated by hypermail 2.3.0.


Sort all by: