Re: IPython cluster plugin questions and suggested improvements

From: Olivier Grisel <no email>
Date: Thu, 18 Oct 2012 10:34:17 +0200

2012/10/17 MinRK <benjaminrk_at_gmail.com>:
>
>
> Sure, if starcluster didn't set up NFS already. You can certainly use
> sshfs, or (much lighter) scp, of course. I'm not sure why you would in the
> context of StarCluster, though, which already has NFS all set up.

I have not checked which mount point is on NFS on the default setup of
starcluster. Is the home folder of each node mapped to the home folder
of the master by default?

Ideally I would only use NFS for small sharing configuration files and
maybe a development workspace with editable source code while having
each node use a the local disk for reading / writing large datafiles
that are dispatched from / saved to S3 buckets at the beginning / end
of the processing.

-- 
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

Received on Thu Oct 18 2012 - 04:34:43 EDT