How fast are your disks?

[Dave Cornutt]

In article <12800 at brl-adm.ARPA> slk at clutx.clarkson.edu (Steve Knodle) writes:
 > Concerning the following discussion:
 > 
 > >From: Tim Bray <tbray at watsol.waterloo.EDU>
 > >Subject: How fast are your disks?
 > >Date: 1 Apr 88 19:09:18 GMT
 >   ...
 > >In a recent meeting we were analyzing the performance of this application that
 > >is rather I/O bound - in particular, it performs a lot of very random accesses
 > >here and there in large (> 100 Mb) files.  Somebody said "Now, we'll assume
 > >that Unix can do a maximum of 30 disk I/O's a second".  Somebody else remarked
 > >that that figure had been remarkably constant for quite some time.  Somebody
 > >else proposed that it was a fundamental law of Computer Science.  (Of course,
 > >we are poor peons restricted to the use of Vaxes and Suns).
 > >
 > >Anyhow - Presumably there are other people out there limited by this
 > >particular bottleneck.  Are there reasonably-priced unix systems out there
 > >that do better?  Are there a set of benchmarks which reliably characterize
 > >system performance in this area?
 >  
 > >Not only is it not a constant, it's not even true.  The sad fact
 > >is most disk controllers for minis/micros are pretty horendous.
 > >Sun's unfortunate use of the Xylogics 450/451 is a prime example.
 > >Anyway, with decent controllers (or multiple controllers) there is
 > >no reason why the figure 30 can't be exceeded and is on decent Unix
 > >systems.
 > >
 > Let me offer, as a point of reference, extracts from simultaneous (vm/io)stat
 > performance logs for our Gould Powernode 9080, which performs very gracefully
 > under severe I/O load, I feel.  The logs were taken during the End-of-Semester
 > Crunch, and are being used to substantiate my request that the machine
 > be upgraded from 8 Meg of memory to 16 Meg.  (The relatively small
 > amount of memory explains the severe paging rate below.)

A few months ago, I got curious to find out how much difference there was
between a Xylogics 450 and some other systems.  I wrote a little program
to write a large buffer out to a raw disk partition a set number of times,
and by timing the program was able to figure out an approximate transfer rate
for the disk subsystem.  To make the contest as fair as possible, I picked out
two systems that were both BSD-based and relatively unloaded at the time I ran
the benchmark.  I tried to get the kernel out of the way as much as possible
by using an unused raw partition for the test (I hope this also minimized the
effects of seek time, since the writes should have been to adjacent cylinders).
I had the program do the write a fairly large number of times to make the
execution time long enough (about 2-5 minutes) to avoid quantization effects
and to make the time needed to load the executable as insignificant as
possible.  I ran this on both a Sun-3 and a Gould PN9080, starting with
a 100k buffer and increasing the size until no further improvement was seen
in the benchmark time.  The exact configurations of the systems were:

Sun-3/160 with 16M memory, Xylogics 450 connected to 2351 Eagle (the older
400M ones, still pretty fast), SunOS 3.2

Gould PN9080, dual processor (this should have had no effect since the
benchmark was just doing I/O), 12M memory, HSDP disk controller connected
to CDC XMD850 (the official name of the 858M disk), UTX 2.0

Both systems were running multi-user but were quiescent at the time.
Neither disk had an active swap partition on it.  I ran vmstat to insure
that no paging or other significant activity occurred during the benchmark.

The Sun topped out at a transfer rate of ~ 700 kB/sec.  This occurred at
a write size of 2M, I believe (I don't remember for sure and I don't have
my notes).  The 9080 hit 2.0 MB/sec and was still getting faster when
I ran out of memory at a write size of 8M.  (I tried a couple of runs at
a 10M write size, and got one result of 2.4 MB/sec, but the system
started paging because I was using all the available physical memory,
and so I was unable to duplicate the result.)

Now, I realize that this was not terribly scientific.  There were any
number of things that could have interfered with the result, such as
differences in the disk driver between SunOS and Gould's UTX.  Also,
Gould systems with HSDPs have their disks set up for a disk sector
size of 1024 bytes instead of the usual 512.  However, this does
reenforce the impression that many people have that the Xylogics 450
is a slow controller.  In defense of the poor, downtrodden 450, I
should say that it is not a bad little board; in fact, it's one of
the nicer Multibus disk controllers on the market, and if I were
putting together a Multibus system, I wouldn't hesitate to buy one.
The problem is that it was never designed to do what Sun is asking
it to do.  It was fine on the Sun-2 line, and it got them over the
hump with the 3/100 machines (there's something to be said for reusing
hardware that you're already familiar with), but it is totally out
of place on a Sun-4 (the VME adaptor probably doesn't help any
either).  Sun needs to come up with a new, native VME controller
to match the faster Sun-4 and 3/200 lines.
-- 
Dave Cornutt, AT&T Bell Labs (rm 4A406,x1088), Holmdel, NJ
UUCP:{ihnp4,allegra,cbosgd}!hotly!dkc
"The opinions expressed herein are not necessarily my employer's, not
necessarily mine, and probably not necessary"