Progress Report

We have begun the work of designing and implementing the self-configuring monitoring system. The first step has been to specify the format of the special request packets that automatically activate monitoring along the network path between communicating endpoints. The monitors will be deployed at the Layer three ingress and egress routers of the ESnet network and within the end site networks. A principal design goal of the system is to provide components that are secure, easy to install, and easy to maintain so that the system does not add a burden to the network's administration. The software components that comprise the Self-Configuring Network Monitor include: a graphical user interface for requesting activation of the monitors; a library that handles activation packets; a monitoring daemon; a data collection and transmission mechanism; and a graphical user interface that displays the monitoring results.

In this first quarter we have designed proof-of-concept prototype versions of each of these components. This prototype monitoring system provides a java interface for users to request activation of the monitor. Currently this interface allows only the source, destination, and port to be specified. The monitor packet is formatted and sent on the network in the direction of the destination traffic. The monitor daemon accepts tcpdump-like filter requests. The daemon watches for an activation packet. Upon receipt of an activation packet, it interprets the packet and then activates a filter for the designated data.

In this first prototype version of the daemon we can only monitor one stream at a time. The headers from the monitored data stream are returned using a TCP connection to the source of the data traffic. The prototype monitor software also contains a graphical interface that can display rudimentary statistics calculated from the returned headers.

One of the core difficulties in this kind of monitoring system is making sure that the monitor is able to capture every packet that it is filtering for and not interfere on the network. To achieve the minimal interference goal, the monitor is designed to operate off of a network tap. To achieve the goal of not losing any of the packets, we have made improvements to the underlying network interface driver to reduce interrupt overhead. The network interface is normally configured to interrupt for every packet but this causes packet loss. We have modified the network driver to coalesce the interrupts based on a timeout. This then brings the loss to zero. In addition, we are prototyping code that will provide accurate timestamps for the packets when they arrive at the monitor.

We have purchased the first four self-configuring network monitor boxes. The machines themselves were ordered with features that match the BRO and NIMI machines. We have completed initial check-out of the machines and installed one on the LBNL DMZ and one in the LBNL booth at SC2001. We used these machines to test our prototype software. The next installations are planned for NERSC, ORNL, and SLAC but are awaiting further testing and development of the software before deployment.