Local / cluster folder bi-directional synchronization

From: Olivier Grisel <no email>
Date: Sun, 20 Jan 2013 00:02:57 +0100

Hi all,

I would like to be able to setup a bi-directional link between the my
home folder in the EC2 cluster and some local folder on my laptop
(e.g. without my working code and ipython notebook under git version
control).

I currently toying with unison [1] and I think I will be able to turn
that into a starcluster plugin.

[1] http://www.cis.upenn.edu/~bcpierce/unison/

I think it should also be quite easy to write a plugin to setup a
reverse ssh tunnel and then use fuse's sshfs command to mount my
laptop folder onto some mount point folder on the cluster (assuming
ssh server is running on my laptop). However the link my break and I
don't want to interrupt a long running computation on the cluster when
I put my laptop to sleep, so the unison approach sounds more
promising.

Before going further, has anybody else worked on similar tools? Any
tips or resources to share?

--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel

Received on Sat Jan 19 2013 - 18:03:19 EST