Minutes of the Performance Monitoring Workgroup
HENP Grand Challenge Collaboration Meeting
1 July 1997

The session began with a brief look at statistics provided by Objectivity's current performance monitoring tools, a discussion of the potential for (and cost of) building monitoring into wrapper classes such as BaBar's BdbRef, and some preliminary discussion of what additional Objectivity hooks might be needed to meet performance monitoring needs. A decision was made to focus attention on measurements that are not specific to Objectivity. Given the limitations of time, the group chose to further focus its attention on single-query performance, while acknowledging that multiuser performance is a critical issue for further discussion.

There seemed to be a consensus that any performance monitoring should include measures of bulk data rates through various components of the architecture, and "meaningful" data rates. A bulk data rate is the rate at which bytes are moved--an example is the number of pages moved, times the page size in bytes, divided by the total time required for delivery. One example of a "meaningful" data rate is the number of events requested, multiplied by the size of an event, divided by the delivery time, but a number of refinements are possible.

Beyond rate measurement, the architecture should be able to track which pages and events are accessed most frequently. Here, frequency refers to the number of queries requesting a page or an event, without weighting according to the number of times any individual query touches a given datum. There was some discussion of the storage implications of maintaining counters at various data access granularities.

The consensus was that one should also aim to track correlations, e.g., which events tend to be requested with which other events. In the "complex query" path of the strawman architecture, one could begin by storing the queries (depending on representation), or the object ids of the events satisfying each query. Analyzing such data with the intention of finding correlations (in this case, for use in potential reclustering) is a typical problem in data mining circles.

It was pointed out that, in the early stages of the Grand Challenge work, there is a natural way to benchmark clustering effectiveness. One can build a database in temporal order (the order in which events are generated by the simulation codes), clustered by class and/or trigger--no attribute-based clustering or cluster analysis--and then recluster the data in a second database. One can then run identical queries against both versions of the database, and compare performance.

Monitoring the efficiency of space utilization (data expansion factors associated with database overhead and index maintenance, for example) was also discussed, but given a lower priority.

Discussion of multiuser performance monitoring was postponed. So, too, was the issue of defining the mix of queries for which performance benchmarks should be obtained, pending additional work by collaboration physicists regarding query workload characterization.

Back to 30 June, 1 July 1997 meeting page.

Last modified: 11 July 1997