gridengine.info

The first thing you need to do is make sure you have the Java^TM platform, the latest JWSDP, Ant, and at least Grid Engine 6.0u7 (or an equivalent maintrunk source build) installed.

I am going to refer to several directories in this tutorial. They are:

Note that you don’t need to have the above set as environment variables in your shell, but if you do, you can just copy and paste commands from this tutorial.

The first step is to create an external bindings file. This file will provide the JAXB class generator with some additional information. Specifically, the external bindings file will 1) assign a package to the generated files and 2) fix a naming conflict in the qstat.xsd schema. Create a file called $SCHEMA_HOME/qstat.xjb with the following contents:

<?xml version="1.0" encoding="UTF-8"?>

<jxb:bindings version="1.0"
  xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
  xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <jxb:bindings schemaLocation="qstat.xsd" node="/xs:schema">
      <jxb:bindings node="//xs:complexType[@name='job_list_t']">
         <jxb:bindings node="//xs:attribute[@name='state']">
            <jxb:property name="stateAttribute"/>
         </jxb:bindings>
      </jxb:bindings>
      <jxb:schemaBindings>
         <jxb:package name="com.sun.grid.xml.qstat"/>
      </jxb:schemaBindings>
   </jxb:bindings>
</jxb:bindings>

This file sets the package to com.sun.grid.xml.qstat, but you can set the package to whatever suits you. Keep in mind, though, that you may want to generate a set of bindings for each of the three schemas, and you don’t want them to overlap.

Next, you’ll generate the binding classes. You do that by running:

% $JAXB_HOME/bin/xjc.sh -d $BIND_HOME -b $SCHEMA_HOME/qstat.xjb $SCHEMA_HOME/qstat.xsd

If all went well, you’ll see a list of the classes that are generated. Congratulations! You now have a qstat XML binding!

What do you do with it? Well, let me get you started. First, for convenience, let’s create an Ant build script to compile the binding and run a sample app. Create a file called $BIND_HOME/build.xml with the following contents:

<?xml version="1.0" standalone="yes"?>
<project basedir="." default="compile">
  <path id="classpath">
    

  <!--compile Java source files-->
  <target name="compile" description="Compile all Java source files">
    <echo message="Compiling the java source files..." />
    <javac destdir="." debug="on">
      <src path="." />
      <classpath refid="classpath" />
    </javac>
  </target>

  <target name="run" depends="compile" description="Run the sample app">
    <echo message="Running the sample application..." />
    <java classname="Main" fork="true">
      <classpath refid="classpath" />
    </java>
  </target>
</project>

This build script is very primative. With any amount of Ant skills, you should be able to write something that better suits your needs. My goal here is only to cover the bare necessities.

Now, let’s create the sample app. Let’s write a simple app that lists the job number of name of all jobs currently in the system, called $BIND_HOME/Main.java. It might look something like this:

import java.util.*;
import com.sun.grid.*;
import javax.xml.bind.*;

public class Main {
    
   public static void main (String[] args) throws Exception {
      // Create a JAXB context
      JAXBContext jc = JAXBContext.newInstance ("com.sun.grid.xml.qstat");
            
      // Use the context to create an Unmarshaller
      Unmarshaller u = jc.createUnmarshaller();
            
      // Fork a qstat -xml
      Process p = Runtime.getRuntime ().exec ("qstat -xml");

      // Let the binding do it's magic
      JobInfo ji = (JobInfo)u.unmarshal (p.getInputStream ());
      List list = ((JobInfoT)ji.getJobInfo ().get (0)).getJobList ();
      Iterator i = list.iterator ();

      while (i.hasNext ()) {
         JobListT jlt = (JobListT)i.next ();

         System.out.println (jlt.getJBJobNumber () + ": " + jlt.getJBName ());
      }
   }
}

In the call to JAXBContext.netInstance(), we specify the package we used to generate the binding. What we get back from the call to Unmarshaller.unmarshal() is an object tree derived from the qstat output. We then walk the object tree until we get to the job list, and then we go through the jobs, printing the number and name for each one.

Clearly, the only way you will know what the object tree looks like is to 1) read the schema, 2) read the generated source files, or 3) generate JavaDocs from the generated source files. 3 is the best option, but 2 is what I actually did.

To build and run this sample app, do the following:

% ant -Djwsdp.home=$JWSDP_HOME build
% ant -Djwsdp.home=$JWSDP_HOME run

If you want to parse the output from qstat -j, you will need to repeat this process with the $SCHEMA_HOME/detailed_job_info.xsd schema. When processing this schema you will need to change the external bindings file a little. You’ll want a file called $SCHEMA_HOME/detailed_job_info.xjb that contains:

<?xml version="1.0" encoding="UTF-8"?>

<jxb:bindings version="1.0"
  xmlns:jxb="http://java.sun.com/xml/ns/jaxb"
  xmlns:xs="http://www.w3.org/2001/XMLSchema">
   <jxb:bindings schemaLocation="detailed_job_info.xsd" node="/xs:schema">
      <jxb:schemaBindings>
         <jxb:package name="com.sun.grid.xml.job"/>
      </jxb:schemaBindings>
   </jxb:bindings>
</jxb:bindings>

Again, feel free to change the package. You’d use this binding the same way you use the other one. There is a third schema in the $SCHEMA_HOME directory, message.xsd, that you won’t likely need, but if you do, you can generate a binding for it just like you did for detailed_job_info.xsd

Clearly, what I have provided here is only a starting point. For more information about using JAXB, see the docs included with the JWSDP download. Particularly useful are the examples. An obvious next step would be to extend the build script to generate JavaDocs and to split the class files out of the source tree. Another good next step would be to customize the binding classes so that, for example, the status code gets returned as a list of strings instead of a binary or’ed int. If you have problems, let me know. This tutorial is still a work in progress, so feedback is welcome.

Scalable Informatics provided the following example code and explanations.

What we use today looks just like this:

use XML::Smart;
my ($xml,$qstat);

$qstat=`/opt/gridengine/bin/lx24-amd64/qstat -xml`;
$xml	= XML::Smart->new($qstat);

foreach ($xml->{job_info}->{queue_info}->{job_list}('@') )
  {
     # stuff with each job.  All the per job attributes are now available as
     # $_->{attribute_name}.
     #
  }

Now if you want to get fancy, and sort by *any* attribute (up or down, using JB_Owner in this case, refer to the XML for what you want to sort

use XML::Smart;
my ($xml,$qstat,@jobs);

$qstat=`/opt/gridengine/bin/lx24-amd64/qstat -xml`;
$xml	= XML::Smart->new($qstat);
@jobs   = $xml->{job_info}->{queue_info}->{job_list}('@');

foreach ( sort { $a->{JB_Owner} cmp  $b->{JB_Owner} } @jobs )
  {
     # stuff with each job.  All the per job attributes are now available as
     # $_->{attribute_name}.
     #
  }

use Date::Manip;
my ($d,$t,$olddate,$delta,$dt,$date);

# ... some place later in the code ...
($d,$t)=split(/\s+/, $_->{JAT_start_time}  );
if ($d =~ /(\d+)\/(\d+)\/(\d+)/)  { $date = sprintf "%.4i%.2i%.2i",$3,$1,$2; }
       if ($t =~ /(\d+):(\d+):(\d+)/)  { $date .= sprintf "%i%i%i",$1,$2,$3; }
       $olddate = ParseDate($date );
$delta = DateCalc($olddate,$today);
      $dt = Delta_Format($delta,0,qw(%st));
       printf  "%.1f second(s)\n",$dt;