Clever job prioritization tip

Posted by chris Thu, 13 Mar 2008 17:28:48 GMT

Grid Engine has a built-in priority mechanism that is useful for allowing end users to sort and prioritize their own personal pending tasks -- this gives the users the ability to submit many jobs but still dictate which of those jobs need to be run more urgently than the rest.

In practice, though, this is actually fairly clunky to implement. By default the following conditions exist:

  • SGE will accept a priority range of -1023 to 1024
  • By default all jobs get assigned a value of 0
  • Only SGE managers can assign priority values higher than 0
  • Normal users can only assign negative priority values
See where we are going here? By default, a non privileged user can only describe some of her jobs as "less important" than others. There is no mechanism (besides granting the user SGE manager authority) for her to say "this job of mine is more important than that other pending job of mine...".

This is, ummmm, awkward to say the least and works in a way that is 100% opposite from what a sensible user or SGE Admin would expect. Users can only decrease the relative priority of their job in the default environment.

A recent mailing list post from Jeff highlights a nice little workaround. Jeff describes creating an entry in the sge_request file that automatically assigns a value of -p -100 to all submitted jobs that don't override the default with their own use of the -p switch.

This is a nice approach because by default it harms nobody (as all jobs have -p -100. Yet it gives headroom for a non privileged user to use the priority range -99 to 0 to designate some of her jobs as more personally important than others.

Background reference: manpage for sge_request.