Progress: April 2003 to July 2003
The focus of the DMF project for has been to lead efforts in the HEP and GGF communities to define requirements for a Grid monitoring system, to finish a prototype GMA implementation, to improve the performance and fault tolerance of NetLogger, and to design and implement a prototype monitoring event archive. More details on each of these topics follow.
The majority of the work this quarter has been on an improving the efficiency of the NetLogger "Activation Service", which can dynamically turn on and off NetLogger monitoring in running applications in response to remote Web Services requests, and allow any number of GMA Consumers to subscribe to a subset of the application's monitoring data. The part of the previous implementation that multiplexed monitoring "events" to multiple consumers did not scale. In the current implementation, the built-in NetLogger filter mechanism is used, greatly increasing efficiency. Part of this task was, of course, finishing and tuning the performance of the filtering mechanism itself. To allow remote "NetLogger filter" queries using the existing GMA/Web Services components, an XML schema equivalent to the filter expression language was added to the prototype GMA implementation, the pyGMA.
We wrote a paper titled " Service "On-Demand Grid Application Tuning and Debugging with the NetLogger Activation Service ", which we submitted to the 4th International Workshop on Grid Computing (Grid2003), and was accepted.
Completed writing the paper for Computing in High Energy and Nuclear Physics (CHEP) conference titled "GMA Instrumentation of the Athena Framework using NetLogger".
We continued to work with the DOE Science Grid and the Atlas Project to define a very early prototype Grid troubleshooting system to let us better understand the issues and requirements of Grid troubleshooting.
We continue to be extremely active in the Global Grid Forum. Brian attended GGF in Portland (Dan was unable to attend for personal reasons), where Brian co-led 2 Network Measurement working group sessions. Brian has been working on a new NMWG document to define a CIM profile for network measurement data. Dan Gunter had a series of discussions with other GGF members about a GMA working group. Brian Tierney continues as the chair of the GGF Nominating Committee, which is responsible for selecting members of the GGF steering committee.
Brian served on the program committee for IEEE High Performance Distributed Computing (HPDC), and organized and lead a panel at HPDC Grids and OGSI. Brian also served on the program committee for the Grid2003 workshop.
We joined the iVDGL Grid3 effort, and will run a NetLogger repository service for Grid3, and help add NetLogger to Globus and Atlas Athena. This will likely be demonstrated at SC2003.
We also started planning for a Grid Troubleshooting analysis demo with the DOE Science Grid for SC2003.
We continue to collaborate with several groups, including NLANR, EU DataGrid, Globus, and the IEPM project at SLAC on the possible use of NetLogger to collect monitoring data for their projects.
We continue working with several groups to help add NetLogger instrumentation to their software. This quarter this includes Bill Allcock and Jenny (ANL), Constantinos Dovorlis (GA Tech), IEPM (SLAC), EU DataGrid R-GMA, and Atlas Athena software.