Storage Manager tested on PDSF

[Arie Shoshani <arie@math.lbl.gov>]

Alex, Henrik, and Luis said that they tested their software
on PDSF and all seems to be working OK.  

What was tested is the following:

1) A test version of a Query Object using Omnibroker (that Alex wrote)
sends a query to the Query Estimator (QE).  

2) Several test versions of Event Iterators (EIs) using Omnibroker
(that Alex wrote) are launched and communicate with the Query Monitor
(QM) using the corresponding query token.  Note: there EIs do not perform
any analysis, but just print out what the EI has received from QM.

3) The QE generates all the (file_ID:event_ID_list)s for each query
and passes it to the QM when Execute command is issued by the Query
Object.

4) The QM keeps track of all EIs requests.  It issues requests to the
Cache Manager to cache (CM) the files.

5) The CM gets from Objectivity the file_name for each file_ID
requested.  It then FTPs the file from HPSS to PDSF disk cache.
(PFTP will be tested later).  It verifies that FTP is successful by
checking that entire file was cached.  If successful, it notifies
Objectivity that the file was cached (ooChangeDB).  It then notifies
the QM that the file was cached.  If not successful, it tried a couple
more times (assuming a transient network failure), and if not
successful, it notifies QM that it failed.

6) The QM selects one of the EIs and notifies it that the file was
cached (passing it file_name).

All the above modules use CORBA communication.  Also, at this time we
assume that the cache is accessed solely by the Storage Manager.  No
coordination with Objectivity caching of files from HPSS is planned for
the current version.
------------

With the above test we found that:

a) Omnibroker and Orbix ORBs communicate OK now for the version tested.
versions are Omnibroker 2.02, 2.0.3 and  2.04, and Orbix 2.3c MT.

b) There is not enough disk cache on PDSF, so we could only work with
a couple of files.
------------

Tests planned (expected to be done this week):

1) The Query Object issues a "quick_estimate" or "full_estimate"
request and get various statistics.  After a "full_estimate" (or after
"execute") it can ask for Event_ID list for events that qualified.

2) Two queries are launched for test purposes -- i.e. two Query Objects
will be launched concurrently.

3) Have EIs send "done_with_file" message to QM.  The QM in turn asks
the CM to purge files.  The CM deletes the file from PDSF cache and
informs Objectivity (ooChangeDB).
-------------

NOTE: We are ready to integrate with the other components as soon as
they are operational (especially, the application that initiates the
Query Object and Event Iterators).  Questions should go to:

Henrik - for Query Estimator (HNordberg@lbl.gov)
Alex - for Query Monitor (ASim@lbl.gov)
Luis - for Cache Manager (LMBernardo@lbl.gov)

Arie.
------------------------------------------------------------------------