It's official: Project Hedeby and ARCo join the SGE codebase

Posted by chris Wed, 13 Dec 2006 15:30:35 GMT

Sun has formally announced the additions promised at SC'06, the full announcement is available online here:

Of the two, ARCo is the more established layered product. This is the SQL driven accounting and reporting tool that was previously only available in the commercial version of N1 Grid Engine from Sun. ARCo uses Java to parse the SGE accounting logs for inclusion into an SQL back-end database. In addition to the metrics found in the accounting logs, ARCo has hooks for calculating useful "derived" metrics that are not explicitly stored in the accounting files.

When I first used ARCo (early on in its very first release version) one of the main weaknesses was the front end web based reporting console - for anything but the most basic reports, a user was expected to paste raw SQL queries into a web form. Sun's act of putting ARCo into the open source codebase should hopefully kickstart an idea that has been floating around for a while -- some sort of community wiki page or repository of user-generated ARCo queries and report templates. ARCo users are encouraged to send these sorts of tips and tricks to the users mailing list.

"Project Hedeby" aka the "Grid Engine Service Domain Management module" also mentioned at SC'06 is at an earlier stage in it's development. The nontechnical description is as follows:

Project Hedeby provides access to a new technology which allows to dynamically manage resources across so called Service Domains. Service Domains can be envisioned as autonomous Grids controlled by a resource manager including but not limited to Grid Engine. Hedeby will adjust the allocation of resources to individual service domains in order to meet Service Level Objectives. Reallocating a host resource to another service domain may include re-provisioning of the underlying virtual or actual operating system stack.

In his interview with GRIDtoday, Fritz provides the following description:

"... provides policy and demand-based re-allocation of arbitrary resources across service domains. Service domains are totally autonomous Grids which are controlled by a workload management facility, such as Grid Engine, but also by arbitrary other service infrastructures like application servers or web servers..."

Thanks to Andy for pointing out that the project codename, "Hedeby" refers to a Viking trade town from the 8th-11th century.