Re: SGE have a job consume all slots?

Contemporary messages sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

From: Chris Dagdigian <no email>
Date: Fri, 24 Feb 2012 10:51:19 -0500

Justin you are correct, what you describe below is the exact way people used to consume more than one job slot (or block off an entire machine for a single non-parallel job back before exclusive host access was available. We often referred to this method on the SGE lists as the "threaded PE" or "threaded PE hack." for people looking for background info.

Dedicated clusters would make a dummy parallel environment called "threaded" or "SMP" for this method but in starcluster just consuming slots from the orte PE should work fine.

My $.02

-Chris

Justin Riley wrote:

Correct me if I'm wrong but if you modify the 'orte' parallel
environment on StarCluster to use the '$pe_slots' allocation_rule:

$ qconf -mp orte
allocation_rule $pe_slots

then according to the SGE docs:

"If the special denominator $pe_slots is used, the full range of
processes as specified with the qsub(1) -pe switch has to be allocated
on a single host (no matter which value belonging to the range is
finally chosen for the job to be allocated)."

So from my understanding after making the above change the following
would use all 8 slots on a single machine for a single job:

$ qsub -cwd -pe orte 8 ./jobscript.sh

HTH,

Chris Dagdigian Principal Consultant, BioTeam Inc.
http://bioteam.net