gridengine.info

For people like myself who are interested (or say, dependent) on the XML output features of Grid Engine it's been a lonely time. This area of Grid Engine was not really getting much love, attention or bug fixes until recently.

Happy to report that this seems to have changed. If you are at all interested in using SGE data in XML form then you may want to:

• Pay attention to this mailing list thread
• Watch this SGE Wiki page

Kudos to Michael Pospisil from the Sun Microsystems SGE developer team in Prague for soliciting and listening to community input -- looks like the change may be bigger than simple bug fixes and output normalization. There is some talk about making XML output more usable to the end-users instead of the current design where XML output is largely a straight representation of internal SGE Cull lists and data structures.

In a follow-up post to Mark's recent announcement we've gotten our hands on some screenshots from Mark showing his tools in use. The screenshots show the results of using XSLT transformations to turn Grid Engine XML data into XHTML form suitable for web pages. The benefit includes web-based visibility into current resource (and software license!) usage. This is exactly the approach that I tried out with the xml-qstat project. Mark is pretty familiar with that effort and will be merging his improvements and enhancements into xml-qstat's SVN repository. Speaking personally as a "scratch an itch" programmer with no real software engineering skill or talent I'm pretty excited to have a real coder take a look at xml-qstat. Related to that I already owe a debt to Petr Jung from Sun who contributed the Java based CommandGenerator code that finally allows xml-qstat to be a 100% Java/Cocoon web application that does not require external perl daemons to cache XML state data.

Before the screen captures, I'd like to ask a favor of people who read this blog. I filed bug Issue #2335 back in July of 2007 and it has not received much love (or even a targeted milestone date for a fix). The bug is a simple one -- "qstat -f -xml" no longer reports load average data which (a) makes xml-qstat a whole lot less useful and (b) breaks the SGE developer philosophy of ensuring that command output returns the same information regardless of output format. Until that bug is fixed it does make sense for xml-qstat to have it's long overdue "1.0" release. If you have a user account over on http://gridengine.sunsource.net I'd appreciate it if you can cast one of your "votes" for Issue 2335. Thanks!

And now the screenshots (edited to mask out personal/company information). Click on each image for a larger version.

qhost overview

Click on through for the rest of the pictures ...

DanT writes:

"...I just published the 0.2 distro of Project Passau on Java.net. Passau is a Java API for accessing the information provided by qstat. It uses the qstat -xml command to produce XML output and then parses that output using a JAXB binding."

Grid Engine 6.x distributions include a "util/resources/schemas/qstat/" directory that currently contains the following files:

• qstat.xsd
• message.xsd
• detailed_job_info.xsd

These are about the best resources one can currently obtain when delving deep into SGE's XML output behavior. They are, however, a bit cryptic to read. Passing the .xsd files through an XML Schema Documentation Generator has resulted in some more human readable output. The translated files can be found here:

• http://gridengine.info/files/qstat.xsd.html
• http://gridengine.info/files/message.xsd.html
• http://gridengine.info/files/detailed_job_info.xsd.html

This topic is one which has been under discussion for some time now. The basic idea is that using the JAXB RI from the JWSDP, we could build a set of classes which would parse qstat output, making it trivial for a developer to write an app which keeps tabs on Grid Engine. I believe the final decision was that we would not officially include such classes with Grid Engine for supportability reasons. Instead, I’m going to explain to you how to build the classes yourself. If you’re too lazy to follow these instructions, here’s a tarball of the classes I generated while writing this post, along with the source, etc.

<?xml version="1.0" encoding="UTF-8"?>

<jxb:bindings version="1.0"
  xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
  xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <jxb:bindings schemaLocation="qstat.xsd" node="/xs:schema">
      <jxb:bindings node="//xs:complexType[@name='job_list_t']">
         <jxb:bindings node="//xs:attribute[@name='state']">
            <jxb:property name="stateAttribute"/>
         </jxb:bindings>
      </jxb:bindings>
      <jxb:schemaBindings>
         <jxb:package name="com.sun.grid.xml.qstat"/>
      </jxb:schemaBindings>
   </jxb:bindings>
</jxb:bindings>

% $JAXB_HOME/bin/xjc.sh -d $BIND_HOME -b $SCHEMA_HOME/qstat.xjb $SCHEMA_HOME/qstat.xsd

<?xml version="1.0" standalone="yes"?>
<project basedir="." default="compile">
  <path id="classpath">
    

  <!--compile Java source files-->
  <target name="compile" description="Compile all Java source files">
    <echo message="Compiling the java source files..." />
    <javac destdir="." debug="on">
      <src path="." />
      <classpath refid="classpath" />
    </javac>
  </target>

  <target name="run" depends="compile" description="Run the sample app">
    <echo message="Running the sample application..." />
    <java classname="Main" fork="true">
      <classpath refid="classpath" />
    </java>
  </target>
</project>

import java.util.*;
import com.sun.grid.*;
import javax.xml.bind.*;

public class Main {
    
   public static void main (String[] args) throws Exception {
      // Create a JAXB context
      JAXBContext jc = JAXBContext.newInstance ("com.sun.grid.xml.qstat");
            
      // Use the context to create an Unmarshaller
      Unmarshaller u = jc.createUnmarshaller();
            
      // Fork a qstat -xml
      Process p = Runtime.getRuntime ().exec ("qstat -xml");

      // Let the binding do it's magic
      JobInfo ji = (JobInfo)u.unmarshal (p.getInputStream ());
      List list = ((JobInfoT)ji.getJobInfo ().get (0)).getJobList ();
      Iterator i = list.iterator ();

      while (i.hasNext ()) {
         JobListT jlt = (JobListT)i.next ();

         System.out.println (jlt.getJBJobNumber () + ": " + jlt.getJBName ());
      }
   }
}

% ant -Djwsdp.home=$JWSDP_HOME build
% ant -Djwsdp.home=$JWSDP_HOME run

<?xml version="1.0" encoding="UTF-8"?>

<jxb:bindings version="1.0"
  xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
  xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <jxb:bindings schemaLocation="detailed_job_info.xsd" node="/xs:schema">
      <jxb:schemaBindings>
         <jxb:package name="com.sun.grid.xml.job"/>
      </jxb:schemaBindings>
   </jxb:bindings>
</jxb:bindings>

Joe Landman from Scalable Informatics posted about his success with the Perl XML::Smart ( CPAN, readme, FAQ, tutorial) module.

Unlike many of the XML handling methods within the Perl universe, this module stands on its own without a huge and complicated chain of external dependencies.

XML::Smart can quickly and cleanly parse XML documents into perl datastructures that can efficiently traversed and sorted. This makes it a great method for simple perl scripts designed to grab bits of data or information that does not get displayed in the human-readble qstat output.

Our 6.0u6 perl based parser fits into a single line, after we grab the data.

	$qstat=`/opt/gridengine/bin/lx24-amd64/qstat -xml`;
	$xml    = XML::Smart->new($qstat);

(no schema/DTD needed)
then for example, iterating over all the jobs ...

	foreach ($xml->{job_info}->{queue_info}->{job_list}('@') )
	  {
	   ...
	  }

Using some example code (included at the end of this article by permission) kindly provided by Joe, I was able to whip up a little "just playing" script that checks all pending jobs for hard resource requests. When a hard request is found, the script simply prints out a line that lists the Job ID, Job Name and the value of the hard resource request. The script looks like this:

#!/usr/bin/perl -w
use XML::Smart;
my ($xml,$qstat);

$qstat=`/opt/sge6s2u1/bin/lx24-amd64/qstat -xml -r -f`;
$xml    = XML::Smart->new($qstat);

foreach ($xml->{job_info}->{job_info}->{job_list}('@') )
{
    if($_->{hard_request}) {
      print "Job ID $_->{JB_job_number} ($_->{JB_name}) has a hard_request: ";
      print "$_->{hard_request}{name}=$_->{hard_request} \n";
    }
}

Output looks like this:

[dag@dcore-amd ~]$ ./test.pl
Job ID 47 (impossibleJob) has a hard_request: arch=darwin 
[dag@dcore-amd ~]$ 

Additional pointers and examples from Scalable Informatics are included below ...

use XML::Smart;
my ($xml,$qstat);

$qstat=`/opt/gridengine/bin/lx24-amd64/qstat -xml`;
$xml	= XML::Smart->new($qstat);

foreach ($xml->{job_info}->{queue_info}->{job_list}('@') )
  {
     # stuff with each job.  All the per job attributes are now available as
     # $_->{attribute_name}.
     #
  }

use XML::Smart;
my ($xml,$qstat,@jobs);

$qstat=`/opt/gridengine/bin/lx24-amd64/qstat -xml`;
$xml	= XML::Smart->new($qstat);
@jobs   = $xml->{job_info}->{queue_info}->{job_list}('@');

foreach ( sort { $a->{JB_Owner} cmp  $b->{JB_Owner} } @jobs )
  {
     # stuff with each job.  All the per job attributes are now available as
     # $_->{attribute_name}.
     #
  }

use Date::Manip;
my ($d,$t,$olddate,$delta,$dt,$date);

# ... some place later in the code ...
($d,$t)=split(/\s+/, $_->{JAT_start_time}  );
if ($d =~ /(\d+)\/(\d+)\/(\d+)/)  { $date = sprintf "%.4i%.2i%.2i",$3,$1,$2; }
       if ($t =~ /(\d+):(\d+):(\d+)/)  { $date .= sprintf "%i%i%i",$1,$2,$3; }
       $olddate = ParseDate($date );
$delta = DateCalc($olddate,$today);
      $dt = Delta_Format($delta,0,qw(%st));
       printf  "%.1f second(s)\n",$dt;