IPython parallel DirectView not distributing to starcluster nodes.

From: Austin So <no email>
Date: Wed, 1 Jul 2015 09:09:35 -0700

I’ve been trying to figure out what I’m doing wrong here, and if it is an issue within the starcluster config file. I’ve exhausted all possible implementations in my code that I could think of.

During set up, 255 engines have been recorded to have been set-up by starcluster upon launch that are available to IPcluster.

Within IPython Notebooks, I’m trying to distribute a function across all my nodes and engines (5_at_r3.8xlarge).

So when the line of code is running, I’m looking at qhost, and I see that only the Master is showing a CPU load. I look at the Cloud Metrics, and I see that only the Master is showing a CPU load. At the suggestion of a friend, I returned a socket.fqdn() call to identify if the results were processed by the master or one of the nodes. All results returned were generated by the Master.

Any hints to identify where the source of the problem lies would be greatly appreciated.

Best

Austin
Received on Wed Jul 01 2015 - 12:09:44 EDT