Grand Challenge Application on HENP Data

Focussed on

Efficient Data Access for RHIC

or

A Sharper Drill Bit for Mining the QGP

Description

The purpose of this project is to develop techniques and tools that will enable efficient access to the massive datasets for the RHIC experiments (beginning in late 1999), in their search for the Quark-Gluon-Plasma (QGP).

The requirements imposed by a very large (PB scale) tape-based database necessitates developing new techniques for organizing data and coordinating access. This involves developing tools to store data as it is generated according to predicted access patterns as well as monitoring access patterns in order to decide when and how to re-organize the data as necessary. A key ingredient to efficient tape access is a disk resident index which is queried to identify which tape files to access, permit scheduling of staging files to disk to balance multi-user access and enable event-parallel processing for analysis of the selected data. These tools will operate with an HPSS storage system and provide data access in an ODMG object framework.

Simulated data generated at NERSC and the RHIC Computing Facility (RCF) will be used in testing and developing these tools before experiment operations begin.

Goals

• Develop tools to store (and re-order) data according to predicted (and observed) access patterns.
• Develop efficient disk resident index with query capability for scheduling and event-parallel processing.
• Develop ODMG compliant interfaces to index, query, and analysis components.
• Generate sample datasets of up to 50TB for testing.
• Demonstrate use of these tools in the mock data challenge(s) before experiment operations begin.
• Provide these tools as part of the RHIC computing capabilities for experiment operations.

Project Participants

This project is associated with the National Energy Research Scientific Computing Center (NERSC) at Berkeley Lab and the RHIC Computing Facility (RCF) at Brookhaven Lab.

• Lawrence Berkeley National Laboratory
  • D. Olson, G. Odyniec, R. Porter, F. Wang, N. Xu, Nuclear Sciences
  • J. Siegrist, I. Hinchliffe, R. Jacobsen, Physics
  • C. Tull, D. Quarrie, W. Johnston, A. Shoshani, D. Rotem, H. Nordberg, Computing Sciences
• Brookhaven National Laboratory
  • B. Gibbard, D. Stampf, J. Flanigan, RCF
  • D. Morrison, T. Wenaus, Physics
• Argonne National Laboratory
  • E. May, D. Malon
• Florida State University
  • G. Riccardi
• Rice University
  • P. Yepes
• University of California, Los Angeles
  • H. Huang
• University of Tennessee
  • S. Sorensen

Contact

For questions and additional information about this project contact Doug Olson (dlolson@lbl.gov).

Related Projects

		CLAS
Computing

GC Project working web.

Last Update: September 17, 1997