Re: loadbalance exit after no jobs

From: David Erickson <no email>
Date: Sun, 26 Feb 2012 22:17:03 -0800

I am using EBS, but when the jobs are over I'm ready to copy the data
back immediately because I have a serial process waiting on it.

-D

On 2/26/2012 9:02 PM, Ron Chen wrote:
> I think it is easy if you are willing to modify the loadbalancer code.
>
> However, why not just mount an EBS, and store the job results there? And when the jobs are done, let the load balancer clean up everything. Then when you finally have time, you go back and start a small instance (or even a micro) and copy data from EBS back to your machine.
>
> If you are not using EBS, then you need to know that if the instance storage fails, all the data is gone.
>
>
> -Ron
>
>
>
> ----- Original Message -----
> From: David Erickson<derickso_at_stanford.edu>
> To: starcluster<starcluster_at_mit.edu>
> Cc:
> Sent: Sunday, February 26, 2012 9:12 PM
> Subject: [StarCluster] loadbalance exit after no jobs
>
> Hi-
> I was reading about the loadbalance command and it looks great, I'd like
> to submit a bunch of jobs (1:1 with VMs) then draw down the VMs once
> jobs end. It looks like loadbalance can do this, one quirk however is
> I'd like the loadbalance command to exit once the queue is empty, but
> NOT kill the master, because I'd like to copy some data off before
> terminating the cluster. Is there any way to do this with the
> loadbalance command? A workaround would be to run loadbalance to handle
> draw down of the cluster and another command watching for all jobs to
> complete which will kill loadbalance once all jobs complete, but it is
> more complex.
>
> Thanks,
> David
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
>
>
>
Received on Mon Feb 27 2012 - 01:18:01 EST