Re: HPSS configuration for GCMDC2

[Henrik Nordberg <HNordberg@lbl.gov>]

Arie Shoshani wrote:
> 
> Dave,
> 
> I have a couple of comments.
> 
> dave zimmerman wrote:
> >
> > This morning I met with John Riordan to discuss the GC Hpss
> > configuration and  the tape situation.
> > I learned that it is possible to lock a tape so that no more files will
> > be written to it.  This facility can be used to maximize the number of
> > tapes available to the GC by transferring say 10 1-Gig files to a single
> > tape, locking it and
> > moving on to the next tape for the next 10 gig of data.  After the
> > GCMDC2 data sinking period is over we would unlock the tapes so that
> > they would be available to be filled out.
> >
> > If this seems reasonable Id like to follow this plan.
> 
> Sounds like a good plan.  But, we should have one tape with enough data
> on it, so that we can run a test where we compare reading all the files
> from a single tape vs. the files spread out on 5 tapes.
> 
> > Regarding the HPSS configuration for MDC2 the proposed configuration is:
> >
> > File purging policy:
> > Do not purge any file which has been accessed in the last 5 minutes
> > Start purge when the cache is 5% full.
> > Stop purge when the cache is empty.
> 
> If we want to empty the HPSS cache between runs, then "start purge"
> should be done when the cache is 0% full, or as close to it as
> possible.  As I recall from MDC1 the smallest % possible was 1%.
> 
> As for the length of time before purging, the longer the better during
> runs, but we need this figure to be low, so that the cache empties
> between runs.  I guess we can make this 10 minutes, so that files that
> will potentially be shared will stay in cache longer.  This means that
> we'll have to wait 10 minutes between runs.  Henrik, Luis: what do you
> think?

10 minutes is not too long. After each run we need to copy the logs and
restart STAC (STorage Access Coordinator, formerly known as SM :) .
We could even set it to 15 or 20 minutes. But even better would be to
have a command to flush the HPSS cache. Is that possible via the 
HPSS API? If so we can have a session on the AIX machine for this
purpose. If not we could make it a macro that sets the time-before-
purge to the minimum (1 min.?), then wait for that time + n seconds,
then set the time-before-purge to some larger value.

 - Henrik
___________________________________________________________
Henrik Nordberg <HNordberg@lbl.gov> http://nordberg.lbl.gov
Scientific Data Management Research Group
Lawrence Berkeley Laboratory