StarCluster - Mailing List Archive

Re: Configuring number of map jobs per cluster node {Hadoop plugin}

From: Rayson Ho <no email>
Date: Thu, 31 May 2012 15:38:58 -0400

While integrating some user contributed Hadoop docs into the Open Grid
Scheduler website, I came across the
"" parameter - a quick Google
search points me to:

Q: I see a maximum of 2 maps/reduces spawned concurrently on each
TaskTracker, how do I increase that?
A: Use the configuration knob:
and mapred.tasktracker.reduce.tasks.maximum to control the number of
maps/reduces spawned simultaneously on a TaskTracker. By default, it
is set to 2, hence one sees a maximum of 2 maps and 2 reduces at a
given instance on a TaskTracker.


Make be it is a matter of setting the parameter??


Open Grid Scheduler / Grid Engine

Scalable Grid Engine Support Program

On Wed, May 30, 2012 at 2:07 PM, Paul McDonagh <> wrote:
> Thanks for creating starcluster, it's great.  I'm using the Hadoop plugin and I'm working on a c1.xlarge instance type. The c1.xlarge type has 20 EC2 Compute units or 8 virtual cores.
> When looking at the job tracking webpages that are set up after the cluster is initiated and running, there is a limit of 2 map jobs per cluster node. How can I alter the number map (or reduce) jobs a particular compute node can run? I can't seem to find how to change this. I'd like to be able to use much more of the compute resources for some of the larger compute instance types.
> Thanks for your help.
> Paul McDonagh
> _______________________________________________
> StarCluster mailing list

Open Grid Scheduler - The Official Open Source Grid Engine
Received on Thu May 31 2012 - 15:39:00 EDT
This archive was generated by hypermail 2.3.0.


Sort all by: