While integrating some user contributed Hadoop docs into the Open Grid
Scheduler website, I came across the
"mapred.tasktracker.map.tasks.maximum" parameter - a quick Google
search points me to:
Q: I see a maximum of 2 maps/reduces spawned concurrently on each
TaskTracker, how do I increase that?
A: Use the configuration knob: mapred.tasktracker.map.tasks.maximum
and mapred.tasktracker.reduce.tasks.maximum to control the number of
maps/reduces spawned simultaneously on a TaskTracker. By default, it
is set to 2, hence one sees a maximum of 2 maps and 2 reduces at a
given instance on a TaskTracker.
Ref:
http://wiki.apache.org/hadoop/FAQ#I_see_a_maximum_of_2_maps.2BAC8-reduces_spawned_concurrently_on_each_TaskTracker.2C_how_do_I_increase_that.3F
Make be it is a matter of setting the parameter??
Rayson
================================
Open Grid Scheduler / Grid Engine
http://gridscheduler.sourceforge.net/
Scalable Grid Engine Support Program
http://www.scalablelogic.com/
On Wed, May 30, 2012 at 2:07 PM, Paul McDonagh <mcdonaghpd_at_gmail.com> wrote:
> Thanks for creating starcluster, it's great. I'm using the Hadoop plugin and I'm working on a c1.xlarge instance type. The c1.xlarge type has 20 EC2 Compute units or 8 virtual cores.
>
> When looking at the job tracking webpages that are set up after the cluster is initiated and running, there is a limit of 2 map jobs per cluster node. How can I alter the number map (or reduce) jobs a particular compute node can run? I can't seem to find how to change this. I'd like to be able to use much more of the compute resources for some of the larger compute instance types.
>
> Thanks for your help.
> Paul McDonagh
>
>
>
> _______________________________________________
> StarCluster mailing list
> StarCluster_at_mit.edu
> http://mailman.mit.edu/mailman/listinfo/starcluster
--
==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/
Received on Thu May 31 2012 - 15:39:00 EDT