Re: ROOT file_ids and Bundle_ids

["David M. Malon" <malon@anl.gov>]

>  
>  This message is maily for Dave M., but it makes assumptions about the 
>  directory structure for MDC2, so it is relevant to others as well.
>  
>  Arie.
>  
>  
>  We discussed the file_id issues and here is what we concluded.
>  
>  In the long run, we'd like to have the Storage Manager software 
>  unafected by the type of files we deal with - Objectivity files, ROOT 
>  files, or any other kind.  To accommodate that, we'll have to use a 
>  consistent set of file_IDs, regardless of where they come from.  I'll 
>  elaborate on how we plan to do that later.
>  
>  As for what we pass to the Event Iterator (EI) for multiple components, 
>  we concluded that we'd like to pass a bundle_id, and when the EI is 
>  done with the bundle, it should use this bundle_id when releasing the 
>  bundle.  However, if there is a value to passing the file_ids (or 
>  file_names) we could do that also at a later time (after MDC2).
>  
>  So, here is what we plan for MDC2 and after MDC2.
>  
>  For MDC2:
>  
>  1)  Both Objectivity files and ROOT files will be pre-assigned 
>  Objectivity file-ids, and those will be used by the index.
>  

   DMM:

   I expressed my concerns last month about faking Objectivity into 
   generating db numbers as fileids for ROOT files, and I thought that 
   Doug and Torre, among others, shared my concerns.  By the way, 
   implementing that deception may involve some subtleties.

   My understanding is that David Zimmerman is working on a method for 
   generating consistent fileids for ROOT and Objectivity files, with 
   a simple interface for use by Storage Manager components to translate 
   from ids to names.  If he does not have time for this, I will volunteer 
   to do it.

>  2)  All file names (including path_names if more that a single 
>  directory is used) will be retrieved from the Objectivity catalog.
>  

   DMM:

   I would be happier if we could amend this to say "will be retrieved 
   from the Objectivity federation."

>  3)  We assume files can be in different directories.  However, the same 
>  directory structure will exist beyond a root-path in HPSS as in Unix.
>  (e.g. suppose the Objectivity root-path is o1/o2/o3 and HPSS root-path is 
>  h1/h2, then a file in Objectivity directory o1/o2/03/x/y/z/file_name, 
>  will be in the HPSS directory as h1/h2/x/y/z/file_name.)  Thus, given the 
>  objectivity path, the HPSS path can be generated.
>  

   DMM:

   I'm not sure I understand this.  Do you mean that I can have an 
   arbitrary number of prefix pairs:

   o1/o2/o3    corresponds to HPSS h1/h2
   o1/o2/o4    corresponds to HPSS h1/h3
   o1/o5/06    corresponds to HPSS h1/h4
   o7/o8       corresponds to HPSS h5/h6/h7
   ..., 
 
   or that I can have only one, so that the HPSS directory structure 
   must (after substituting prefixes) precisely mirror the disk directory 
   structure?
   
>  4)  The QM will pass to the EI only the bundle_id and the set of 
>  event_ids that qualify for this bundle.  The EI will return this 
>  bundle_id when releasing the bundle.
>  

   DMM:

   Can Alex or Henrik comment on this:  How difficult would it be for MDC2 to 
   support an interface that takes a bundle_id as input and returns the 
   corresponding filelist (fileids or filenames)?  If this is done, I think 
   I can live with the bundle_id approach for the OOI in MDC2.  

>  
>  After MDC2:
>  
   DMM:

   ... I'll wait to comment on post-MDC2 ideas.

   Cheers,

   David