Re: loadbalance exit after no jobs

From: Ron Chen <no email>
Date: Mon, 27 Feb 2012 19:08:13 -0800 (PST)

I think it is a common problem -- so I think the Elastic Load Balancer should be extended to support a shutdown script.

Not saying that Dustin's suggestion of using a plugin is not good, but I think a lot of times users don't want to learn to write a plugin (reinvent the steering wheel) just to handle data copying and/or cluster shutdown.


 -Ron



----- Original Message -----
From: Dustin Machi <dmachi_at_vbi.vt.edu>
To: David Erickson <derickso_at_stanford.edu>
Cc: Ron Chen <ron_chen_123_at_yahoo.com>; starcluster <starcluster_at_mit.edu>
Sent: Monday, February 27, 2012 7:18 AM
Subject: Re: [StarCluster] loadbalance exit after no jobs

The other straight forward alternative would be to create a plugin that executes on the masternode when it is shutdown.  This way you would let the master node be killed, but it will execute this plugin to copy your files before it shuts down.

Dustin

On Feb 27, 2012, at 1:17 AM, David Erickson wrote:

> I am using EBS, but when the jobs are over I'm ready to copy the data
> back immediately because I have a serial process waiting on it.
>
> -D
>
> On 2/26/2012 9:02 PM, Ron Chen wrote:
>> I think it is easy if you are willing to modify the loadbalancer code.
>>
>> However, why not just mount an EBS, and store the job results there? And when the jobs are done, let the load balancer clean up everything. Then when you finally have time, you go back and start a small instance (or even a micro) and copy data from EBS back to your machine.
>>
>> If you are not using EBS, then you need to know that if the instance storage fails, all the data is gone.
>>
>>
>>  -Ron
>>
>>
>>
>> ----- Original Message -----
>> From: David Erickson<derickso_at_stanford.edu>
>> To: starcluster<starcluster_at_mit.edu>
>> Cc:
>> Sent: Sunday, February 26, 2012 9:12 PM
>> Subject: [StarCluster] loadbalance exit after no jobs
>>
>> Hi-
>> I was reading about the loadbalance command and it looks great, I'd like
>> to submit a bunch of jobs (1:1 with VMs) then draw down the VMs once
>> jobs end.  It looks like loadbalance can do this, one quirk however is
>> I'd like the loadbalance command to exit once the queue is empty, but
>> NOT kill the master, because I'd like to copy some data off before
>> terminating the cluster.  Is there any way to do this with the
>> loadbalance command?  A workaround would be to run loadbalance to handle
>> draw down of the cluster and another command watching for all jobs to
>> complete which will kill loadbalance once all jobs complete, but it is
>> more complex.
>>
>> Thanks,
>> David
>> _______________________________________________
>> StarCluster mailing list
>> StarCluster_at_mit.edu
>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>
>>
>>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
Received on Mon Feb 27 2012 - 22:08:15 EST