Re: HPSS access from PDSF

[Luis Bernardo <bernardo@math.lbl.gov>]

Craig,

I don't know whether this is relevant but the HPSS User's Guide says (pag
1-3): "Each COS describes a desired service in terms of characteristics
such as minimum and maximum file size, transfer rate, access frequency,
latency, and valid read or write operations."
Did you check whether the transfer rate depends on the file size? (What 
happens if the file size is not the right one for that Class Of
Service?). And maybe even better, do you know where to get the
characteristics for each COS?

Luis


On Sat, 25 Jul 1998, Craig E. Tull wrote:

> HPSS-Users;
> 
> I have recently finished some tests of the network connectivity between the
> PDSF room and the NERSC Mass Sortage system (HPSS). These tests have been
> conducted using the PDSF STAR cluster (starsu00, starlx01-08) since they
> have 100bT connections to the PDSF network switch and are fast enough to
> give reasonable results.
> 
> SOLARIS <<<<<<<<<<<<<<<<<<
> 
> starsu00 -> HPSS COS1 : write using pftp/pput
> 	2.12 MByte/sec
> 
> starsu00 <- HPSS COS1 : read using pftp/pput
> 	2.25 MByte/sec
> 
> starsu00 -> HPSS COS3 : write using pftp/pput/explicit COS
> 	4.07 MByte/sec
> 
> starsu00 <- HPSS COS3 : read using pftp/pget
> 	5.75 MByte/sec
> 
> LINUX <<<<<<<<<<<<<<<<<<<<<
> 
> starlx06 <- HPSS COS3 : read using ftp/get
> 	4.40 MByte/sec
> 
> The lesson here is this:
> 
> The large Class Of Service (COS1) is much slower than the medium Class Of
> Service (COS3) for HENP-like access. After speaking with Nancy Meyer, I
> believe the reason is two-fold:
> 
> 1> The connection between PDSF and the HPSS goes through a 10 bT connection
> to access COS1.
> 2> The HIPPI mounted MaxStrat disk cache is very heavily used and can form
> a bottle neck (ie. while your job waits in a queue with Cray jobs).
> 
> - Craig
> 
> 
> -----------------------------------------------------
>  Craig E. Tull, Ph.D.
>  National Energy Research Scientific Computing Center
>  E.O. Lawrence Berkeley National Laboratory
>  EMail: cetull@lbl.gov       WWW: http://iago.lbl.gov/
>  Phone: (510)486-7253        FAX: (510)486-4004       
> 

--------------------------------------------------------------
Luis M. Bernardo  <lmbernardo@lbl.gov> <www.lbl.gov/~bernardo>  
Scientific Data Management Research & Development Group
Lawrence Berkeley National Laboratory         
Berkeley, CA 94720