STACS scalability plans

[Arie Shoshani <arie@math.lbl.gov>]

1.  Support for flexible OID type definitions [flexibility]
	   (treat OIDs opaquely)
	-- effect QE, EI (maybe QM)

2.  OIDs per bundle available on demand from QE [scalability]
	-- QE stores OID-list per bundle on disk and provides 
	    get_OID_per_bundle function
	-- OIDs not passed to QM
	-- QM gets OIDs when bundles are passed to EI.
	-- alternative: EI gets OIDs from QE using bundle-ID

3.  Catalog builder [scalability + flexibility]
	-- Assigns STACS_internal FIDs to files
	-- maps FIDs to local_fname, remote_fname (hpss), file_size
            (tape_id - future)
	-- will use prefix compression
	-- provide get_local_fname, get_remote_fname (given FID)
	-- provides get_FID (given local_fname)
	-- get_file_size (given local_name), used by QM
	-- effect on QM minimal, CM to change calls
	-- index builder will use FIDs
	-- index feeder will use get_FID (Dave Z.)
	-- to be implemented by Luis

4.  Queue stage requests [robustness]
	-- CM will maintain queue
	-- QM-QE: support multi-file stage request
	-- QM-QE: support multi-file abort request
	-- CM: support dynamic PFTP limit to HPSS

5.  Moving large chunks of data through CORBA interfaces [scalability]
	-- check transfer size with ORBIX, TAO, Orbacus
	-- implement one_block_at_a_time capability
	-- effect QO-QE: on returning OID_list
	-- effect QE-QM: on passing bundle_list per query

6.  Concurrent thread limit on CORBA products [scalability]
	-- Check limits
	-- effect QM, QE, Maybe EI

7.  Memory limitations [scalability]
	-- QE: store bit_sliced files in disk, swap as needed
	-- QM: store dynamic structures to disk, swap as needed
	-- Optimize QM structures

8.  Other items
	-- QM on rmds03 for many queries
	-- Administrative IDL to QM
	-- QM Policy change: prefetch only if space > parameter