Re: executing plugins before enabling/adding to queue

From: James Ching <no email>
Date: Sat, 27 Jun 2015 18:17:27 -0400

so I should include "sge" at the end of the plugin list? I tried that but
no luck, I think the actual plugin is called "clustersetup" when I see the
status of adding a new node, that is what adds the node to SGE? I tried
adding that to the end of the plugins and it gave me an error, is it called
something else?

On Sat, Jun 27, 2015 at 6:13 PM, Lyn Gerner <schedulerqueen_at_gmail.com>
wrote:

> If you disable_queue=true, and also have your own plugin in front of
> (preceding) the sge plugin in the plugins= line for your cluster config,
> then your plugin should run first.
>
> On Sat, Jun 27, 2015 at 12:07 PM, James Ching <evolve20_at_gmail.com> wrote:
>
>> Thank you for the reply Lyn, yes I do have a def on_add_node, everything
>> works fine with the scaling action, there are no failures. The problem I
>> have is that, the node is added to the SGE queue before the formatting
>> happens. The built-in plugin "clustersetup" executes first before any
>> other plugins get executed so the node gets added and starts accepting jobs
>> before the ephemeral drive gets formatted which causes my jobs to fail
>> because it's looking for the local drive. I do not think there's a way to
>> set the clustersetup plugin to run after custom plugins which would be
>> ideal but maybe I have to customize my plugin to disable the node format
>> then enable? There may be a better way?
>>
>> On Sat, Jun 27, 2015 at 5:54 PM, Lyn Gerner <schedulerqueen_at_gmail.com>
>> wrote:
>>
>>> Hi James,
>>>
>>> Your high-level process seems correct. Do you have a "def on_add_node"
>>> line in your plugin?
>>>
>>> If yes, it's probably best if you post your config, plugin, and the log
>>> from your failed scaling action.
>>>
>>> Best,
>>> Lyn
>>>
>>>
>>> On Sat, Jun 27, 2015 at 10:06 AM, James Ching <evolve20_at_gmail.com>
>>> wrote:
>>>
>>>> Re-posting this question, any thoughts on how I may be able to have the
>>>> clustersetup plugin run after a custom plugin? I have jobs that rely on
>>>> the local space hence why I want the ephemeral drives formatted first
>>>> before dropping it into the queue to take jobs.
>>>>
>>>> Thanks in advance!
>>>>
>>>>
>>>> On Wed, May 27, 2015 at 10:29 AM, James Ching <evolve20_at_gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm currently using a plugin to format the local ephemeral drives and
>>>>> it works great when I launch a new cluster but when I use the load balance
>>>>> feature and it adds nodes to expand the queue, the node being formatted
>>>>> takes in jobs and they end up failing because it's in the process of
>>>>> formatting the drives.
>>>>>
>>>>> I tried the queue disable_queue feature and it doesn't seem to work, I
>>>>> also ensured that when I declared PLUGINS, i have the format first, then
>>>>> sge last but still no luck... is there something that I am missing to have
>>>>> the node added to the queue after the plugins are executed?
>>>>>
>>>>> Thanks!
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> StarCluster mailing list
>>>> StarCluster_at_mit.edu
>>>> http://mailman.mit.edu/mailman/listinfo/starcluster
>>>>
>>>>
>>>
>>
>
Received on Sat Jun 27 2015 - 18:17:31 EDT