Methodology

From NetLogger

The main tenets of the methodology are:

1. Instrument as many components as possible to collect both end-to-end and top-to-bottom data. These components include application software, middleware, operating system, and networks. The more components that are instrumented the better: the name of the game is correlation.
2. All monitoring events must use a common format and common set of attributes, as described in our "Logging Best Practices" document. Convert other formats into this common format. Monitoring events most also all contain a precision timestamp globally synchronized via a clock synchronization method such as NTP.
3. By default, have coarse-grained logging of entry and exit from applications and major components. At higher debug levels, include: entry and exit from any program or software component, and begin/end of all I/O (disk and network).
4. As much as possible, collect all log data in a central location. Load it into a relational database.
5. Use event correlation and visualization tools (not just grep and vi) to analyze the monitoring event logs.