Hope you are doing well

1) You can easily identify the long running map-reduce jobs from the WebUI of Jobhistory server daemon

To start the Jonhistoryserver daemon in a node run the command below :

./sbin/mr-jobhistory-daemon.sh start historyserver

Now To access the history server:

http://ip_address_of_jobhistory_server_node:19888


2) Method To Tune Up the Long Running Jobs:

    Change the Priority of a Job:

hadoop job -set-priority <job-id> <priority>


Hadoop Job Priority Levels:
There are total 5 different priority level are there. These priority levels are listed below:

  • VERY_HIGH
  • HIGH
  • NORMAL
  • LOW
  • VERY_LOW

From the names of the priority levels, it is quite obvious that highest priority is given to the jobs whose priority level is VERY_HIGH and least priority is given to the jobs whose priority level is VERY_LOW

Example:

In the following example, the priority of the long running job is set to VERY_LOW

hadoop job -set-priority job_201403160138_54485 VERY_LOW


I hope this will resolve the query

Pelase let me know if you have any further issue regarding this.

Feel free to contact us in case you have any query.