Re: New Grid Engine Hadoop Integration HOWTO
This archive was generated by
This and the previous email are a couple of really good suggestions. I'll try 'em out and see what happens.
On Jun 1, 2012, at 14:52, Rayson Ho wrote:
> If you are running Hadoop on StarCluster, you may also be interested
> in this new method contributed by Prakashan Korambath of UCLA.
> The difference between the original SGE 6.2u5 method vs the new one is
> that with Prakashan's approach, Grid Engine is used for resource
> allocation, and the Hadoop job scheduler/Job Tracker is used to handle
> all the MapReduce operations. A Hadoop cluster is created on demand
> with Prakashan's approach, but in the original SGE 6.2u5 method Grid
> Engine replaces the Hadoop job scheduler.
> As standard Grid Engine PEs are used in this new approach, one can
> call "qrsh -inherit" and use Grid Engine's method to start Hadoop
> services on remote nodes, and thus get full job control, job
> accounting, and cleanup at terminate benefits like any other tight PE
> Open Grid Scheduler / Grid Engine
> Scalable Grid Engine Support Program
> StarCluster mailing list
Received on Fri Jun 01 2012 - 15:31:57 EDT