/disk0, was MDC1-GC:lessons learned

[Shigeki Misawa <misawa@bnl.gov>]

If the 2 MB/s transfer rate is to /disk0 on rmds03, I believe I have
tracked down the source of the problem.

At some point, the SCSI bus that connects the drives in the /disk0
array renegotiated the transfer rates between the SCSI controller on
rmds03 and the individual drives. Some drives are at 10MB/sec, others
are at 20MB/sec, some are running SCSI Narrow, others are running SCSI 
Wide. None are running at their rated 40MB/sec.

As a result of this asymmetry, Online Disk Suite and the SCSI device
driver was left with the complex task of reconstructing the stripe
writes and reads. This explains the high utilization and low
read/write performance reported by the ODS monitoring tools and
zoom.se.

Unfortunately, I didn't think of this earlier and only just today
installed the tool that verified the problem.

At this point, I know of no tools that will reset the SCSI bus and
force renegotiation between the drives and the SCSI controller. As a
result, the problem will only be fixed when the system is
rebooted. (Note that the problem only affects /disk0)


Sorry about that.

Shigeki



 > > 3. Transfer rate between caches
 > > 
 > > Observation: most of the time we got about 2 MB/s.  Sometimes we got as
 > > much as 5 or 6, but we observed often .5 to 1 MB/s.
 > > Reason: network is shared.
 > 
 > Are you sure it's not a configuration problem? I don't think we got
 > rates at 20kB/s becuase the network was shared.
 >