-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi Jonathan,
The plugin still needs documenting but here's the basic flow:
Put the plugin in the config:
[plugin ipcluster]
setup_class = starcluster.plugins.ipcluster.IPCluster
Then put the plugin in a cluster template:
[cluster smallcluster]
...
plugins = ipcluster
Then start a cluster:
$ starcluster start myipcluster
If you didn't put the plugin in the default cluster template (specified
by default_template in the [global] section) you can specify the
template like so:
$ starcluster start -c smallcluster myipcluster
This will start a cluster and run the ipcluster plugin which will setup
an ipcluster session for your CLUSTER_USER (defaults to sgeadmin). Then
to use ipcluster:
$ starcluster sshmaster -u sgeadmin myipcluster
sgeadmin_at_master~ ipython
[~]> from IPython.kernel.client import MultiEngineClient
[~]> mec = MultiEngineClient()
[~]> mec.get_ids()
[0, 1]
In the above IPython session we see that we have two nodes in our
ipcluster. In general, you will have as many clients as you have nodes
in your StarCluster.
After you have a MultiEngineClient instance you can then do things like:
[~]> mec.map(lambda x: x**10, range(32)) # happens in parallel
[~]> mec.execute('d=10; print d') # executes on all nodes in parallel
For more details see:
http://ipython.org/ipython-doc/rel-0.10.2/html/parallel/parallel_multiengine.html#quick-and-easy-parallelism
HTH,
~Justin
On 07/26/2011 11:56 PM, Jonathan Taylor wrote:
> Hi,
>
> I was wondering if someone had any examples / documentations on how
> the IPCluster plugin works in StarCluster.
>
> Thanks,
>
> Jonathan
>
> -- Jonathan Taylor Dept. of Statistics Sequoia Hall, 137
> 390 Serra Mall Stanford, CA 94305 Tel: 650.723.9230 Fax:
> 650.725.8977 Web: http://www-stat.stanford.edu/~jtaylo
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla -
http://enigmail.mozdev.org/
iEYEARECAAYFAk4wY50ACgkQ4llAkMfDcrl5IwCeINhKGiM7nQgIEN7Bec0cQsmJ
yn4An3kpz6CPlxfNt0t/tHKtofPG9Nwy
=jnrj
-----END PGP SIGNATURE-----
Received on Wed Jul 27 2011 - 15:14:39 EDT