Tests for MDC2

[Arie Shoshani <arie@math.lbl.gov>]

Tests planned for MDC2

This is a summary of discussions I had with Alex, Henrik, and Luis.
Anybody interested, please take a look and propose changes and/or
additional tests you think may be useful.  We should also discuss this
during our meeting on Friday.

Similarly to MDC1, we propose to run tests that verify:
1) Robustness
2) Effectiveness of caching policy
3) Effectiveness of query overlap
4) Effectiveness of event clustering
5) Effectiveness of using multiple tape drives
6) Objectivity overhead

Some details on each are given next.

1) Robustness: we plan 3 types of tests:

1a) Run long runs - as long as practical, such as running overnight.
1b) Run lots of queries - these could be variations on queries
    provided (we need .some typical multi-component queries.  Will 
    Dave z. provide that?)
1c) 2-3 queries with lots of events (requiring the caching of 100's of
    files)

2) Effectiveness of caching policy

We plan to run the same query twice (or more times) WITHOUT caching
policy, and compare that to runs WITH caching policy turned on.  The
run without caching policy will still coordinate the caching of all
files in a bundle, but will cache the bundles in the order provided by
the QE.

3) Effectiveness of query overlap

To perform such test we need to be able to specify for each query what
files are involved in bundles.  For this purpose, we'll need to use
the index on file_IDs.

4) Effectiveness of event clustering

For this measurement we'll need to control how many files we cache for
a query.  One test is to get a predetermined number of events from a
large number of file (30-50).  Then, we will run a query with the same
number of events in fewer files (5-10).

To be able to do that we used in MDC1 the "tagrand" values that were
provided by Dave Z.  We used that to index on "tagrand" and use it in
query designs.  (Dave Z., can we expect "tagrand" per event for MDC2?)

5) Effectiveness of using multiple tape drives

The easiest way of testing that is to put all component1 files on one
tape, all component2 files on a second tape, etc.  We do not know to
what extent this is so.  Then running queries that have components in
different files will demonstrate the effectiveness of multiple drives.

We may also want to have the EI issue more than one bundle
pre-fetching (or cache ahead) request.  Depending on the distribution
of files on tapes we may be able to show the effectiveness of multiple
drives. 

6) Objectivity overhead

We hope that for MDC2, it will be possible to run the User Code (Dave
M.'s simulation) and real analysis code (using STAF) on a linux
machine.  This will permit the Storage Manager to measure the effect
of Objectivity moving files over the network.  This will tell us
whether the real setup with Linux machines over the network is
introducing unexpectedly large delays.

This type of measurements will show the combined effect of Objectivity
overhead and network speed.  Measuring each separately will require
some additional logs done by the EI (Dave M, do you have ideas here 
on what should be logged?)

The above measurements will be compared with the UC or SII running on
the main server (the E4000).


Comments?

Arie.