While integrating some user contributed Hadoop docs into the Open Grid
Scheduler website, I came across the
"mapred.tasktracker.map.tasks.maximum" parameter - a quick Google
search points me to:
Q: I see a maximum of 2 maps/reduces spawned concurrently on each
TaskTracker, how do I increase that?
A: Use the configuration knob: mapred.tasktracker.map.tasks.maximum
and mapred.tasktracker.reduce.tasks.maximum to control the number of
maps/reduces spawned simultaneously on a TaskTracker. By default, it
is set to 2, hence one sees a maximum of 2 maps and 2 reduces at a
given instance on a TaskTracker.
Make be it is a matter of setting the parameter??
Open Grid Scheduler / Grid Engine
Scalable Grid Engine Support Program
On Wed, May 30, 2012 at 2:07 PM, Paul McDonagh <mcdonaghpd_at_gmail.com> wrote:
> Thanks for creating starcluster, it's great. I'm using the Hadoop plugin and I'm working on a c1.xlarge instance type. The c1.xlarge type has 20 EC2 Compute units or 8 virtual cores.
> When looking at the job tracking webpages that are set up after the cluster is initiated and running, there is a limit of 2 map jobs per cluster node. How can I alter the number map (or reduce) jobs a particular compute node can run? I can't seem to find how to change this. I'd like to be able to use much more of the compute resources for some of the larger compute instance types.
> Thanks for your help.
> Paul McDonagh
> StarCluster mailing list
Open Grid Scheduler - The Official Open Source Grid Engine
Received on Thu May 31 2012 - 15:39:00 EDT