Summary notes from HENP GC workshop, 2-3 Oct. 1997, BNL.

By D. Olson (updated 10/10/97)

The agenda, summary information and some slides from the meeting are available at http://www-rnc.lbl.gov/GC/meetings/2oct97/default.htm.

Contents

Introduction – D. Olson

RHIC Event Store Task Force activities – T. Wenaus

MDSSim: simulation of MDS – D. Stampf

Storage manager – A. Shoshani

SBIR: A New Secondary Access Mechanism for Indexing Very Large Data Sets – Shashi Das

Software components for RHIC – D. Olson

STAF to Storage Manager interface – C. Tull

Mock Data Challenge: STAR – T. Wenaus

Mock Data Challenge: PHENIX – S. Sorensen

Schedule: RHIC requirements for features of the GC project – D. Olson

NERSC, RCF environment for MDS-CAS development – C. Tull, C. Price

Plans, Action Items – D. Olson

Attendees

Introduction – D. Olson

The focus of this workshop is to identify the specific components that the grand challenge effort will contribute to the RHIC data access and analysis implementation as well as the schedule for which this functionality is required.

The first day of the workshop is devoted to clarifying the description of the components and the second is devoted to the schedule.

For the FY98 allocation at NERSC, it is doubtful that very much can be used until the port of the subset of CERNlib to the T3E is functional.

RHIC Event Store Task Force activities – T. Wenaus

Broad consensus apparent, outside of PHOBOS, that Objectivity is seen as offering a solution that credibly meets the requirements we have developed and ROOT does not offer a credible case based on present information.

While a consensus on a role for Objectivity is becoming clear, the details of how Objectivity will best be used (at startup and later on) are not clear, and won't be on a time scale of Nov 1. The detailed plan for Objectivity's usage and role will evolve in the course of the development process that can begin in earnest once this task force deals with the technology decision.

MDSSim: simulation of MDS – D. Stampf

A simulation of how users interact with the MDS is being developed. The simulation (written in Java) has users, queries, memory manager, disk store and tape library.

• Queries - notify memory manager of integrated user requests
• Memory Manager - "Optimizes" disk access and manages the disk store
• User - Access events ala iterator technique

There is a large overlap between this simulation and the storage manager described by Arie. It is important to clarify who will actually implement which pieces so there is a coordinated effort.

Storage manager – A. Shoshani

Some results from cluster analysis of the NA49 data were shown. A tool which shows the distribution of population in fixed-size cells was described as potentially useful to physicists.

The interface between the storage manager and STAF was presented. The storage manager components are a query estimator, a query monitor and a cache manager. The components of STAF are the event iterator and query object. The event iterator receives references to events contained within the object manager.

SBIR: A New Secondary Access Mechanism for Indexing Very Large Data Sets – Shashi Das

This DOE funded phase-1 SBIR project is directed at developing a technique for building an index to physics data where ranges of property values are mapped onto bit values so that very fast comparisons can be made against a selection criteria.

Software components for RHIC – D. Olson

The set of software components to be developed were reviewed in the context of an updated RHIC analysis architecture diagram. The storage manager components (from LBL) are the query estimator, query monitor, cache manager and utilities. The analysis framework components (from LBL & ANL) are a query object, event iterator and parallel processing interface. The object manager component to be used will be decided in the context of the RHIC event store task force. The load balancing component (from RCF) provides the overall facility resource management.

STAF to Storage Manager interface – C. Tull

The analysis side of the interaction with the storage manager was described. The event iterator handles the run-time interaction with the query monitor. On the user side it provides a simple looping over all events that satisfy the query. At the lower level, it interacts with the query monitor as each cell (file) is moved into the disk cache.

Mock Data Challenge: STAR – T. Wenaus, PHENIX – S. Sorensen

The first mock data challenge for both STAR and PHENIX will occur during summer 1998. They will consist of a complete processing chain from simulations through event reconstruction and analysis. PHENIX estimates 100K events in the first challenge.

Schedule: RHIC requirements for features of the GC project – D. Olson

The schedule was discussed in view of the functional requirements necessary for the mock data challenges leading up to real data at RHIC. Both STAR and PHENIX had essentially the same schedule for requirements. These are (ordered by time):

11/97:
first STAF-object manager interface

12/97:
first detailed interface to query monitor and cache manager defined
object design policies for mapping event data onto object manager
query language defined

6/98:
HPSS interface for controlling file placement
queuing & load balancing
first implementation of query monitor, cache manager
query estimator gives estimate for the number of events that satisfy the query

1/99 (mock data challenge 2):
parallel event processing
query estimator gives more detailed estimate of time and other resources
HPSS interface includes file location for ordering reading of files

9/2000:
cluster analysis and possible re-structuring of real data

NERSC, RCF environment for MDS-CAS development – C. Tull, C. Price

The FY98 NERSC environment and interface planned between HPSS and PDSF was described and compared with the FY98 RCF plans for MDS-CAS. It appeared that software development as an HPSS client interface on Sparc Solaris should have reasonable compatibility between NERSC and RCF.

Plans, Action Items – D. Olson

A decision about the distributed messaging infrastructure (like CORBA ,and a particular vendor) needs to be made fairly soon. Probably Orbix is a good candidate since it being used by PHENIX on-line and likely in STAR on-line.

A workshop with a few hands-on people to define more of the detailed functionality and interfaces between the components of the RHIC analysis architecture is planned at LBL for the week of Nov. 1.

An Objectivity "Quick Start" class where we can work with 1 or 2 Objectivity technical people about how to mape our problem onto Objectivity and develop a prototype is targeted for the second or third week of December.

Attendees