gridengine.info

Ever wonder why IO usage for Grid Engine jobs running on Linux systems are not captured in either the SGE accounting or reporting logs?

This message posted to the Users mailing list kicked off an interesting thread and even generated a new Enhancement Issue and submitted patch.

It turns out that IO usage is always reported as "0.00000" under Linux because the built in PDC code within Grid Engine does not have an easy way (under Linux) to learn about IO consumption on a per-task or per-process basis.

Some additional digging by the original poster revealed some interesting Linux kernel options:

The Linux kernel can be compiled with CONFIG_TASKSTATS and CONFIG_TASK_IO_ACCOUNTING options which enable simple per-process I/O usage tobe counted through /proc/(PID)/io as well as the taskstats interface. The execd's PDC module is not aware of these interfaces, and therefore makes no attempt to count this usage under Linux.

In Issue 2429 a patch is submitted that lets the SGE PDC code be aware of io reporting values that can be found in /proc/(PID)/io.

How you can help:

• For your particular flavor of Linux, determine if the kernel options "CONFIG_TASKSTATS" and "CONFIG_TASK_IO_ACCOUNTING" are enabled in the default vendor supplied kernel. Add this data as a comment on Issue 2429.
• Test out the patch yourself