more detailed agenda items for meeting at ANL

[Arie Shoshani <arie@math.lbl.gov>]

Alex, Henrik, Luis, and I discussed soem items we think should be
included in the agenda for our next meetings.  here they are:

1.  File index for non-objectivity databases

Proposal: we plan to add an index for this purpose to the QE.  Each
file will be assigned an internal FID at the time of index creation.
The index will map each FID to unix_file_name and hpss_file_name.
The index will use prefix-compression on the paths of the file names.
The QM and CM will use this index (calling QE) to get the file names.

This item will also eliminate the need to recover from Objectivity
failures (e.g. lockserver down), since STACS will not communicate
with Objectivity at all.  But, for Objectivity files the index will
have to keep Objectivity FIDs as well.

Henrik will discuss this item.


2.  Scalability tests

We think the following are good things to scale:

2a. Lots of queries (100-200) - easy to setup with scripts we already
used in MDC2.

2b. Lots of PFTPs (100-200) - easy to setup, 10 queries each
requesting 2 bundles with 7 componenets each (that are mutually
exclusive), generate 140 PFTPs.  Plans are for the CM to keep PFTP
queues beyond what HPSS will handle, and resubmitting requests till
they are fulfilled.

2c. Lot of files/events per query - we need to discuss how.  Check
effect on returning large event lists to QO (say 1 million).  Check
effect on QE passing to QM of large bundle lists (say 10,000 file).
Also, we need to check how much memory and time QE and QM use for
processing of each query.

2d. Lots of bundles that are not perfectly alligned (as in Hijing) -
we need to discuss how.

2e. Lots of components -- can't think of why we need to go beyond 20
or so.  Discuss.


3.  Other issues to discuss

3a. How do we handle event_ids and component_ids in root?  Currently,
root uses a sequence number into a file.  Do we use that for
object_ids?  Should we assume that applications using root file will
access a tag database?

3b. Should we plan to handle mixed Objectivity/root files?  What do
we return the EI?  File names for root, fids for Objectivity?

3b. How should we handle files that stay permannetly in cache?  Do we
get this information at the time of index building?  Will queries
refer to files that are permanently in cache?


4.  Pre-fetching with pre-emption

Preemption is the removal of pre-fetched files from cache to make
room when needed by another query.  How valuable is this feature?
How do we decide what to preempt?