When is a 68020 faster than a MIPS R2000?

[Jim Helman]

The May issue of IEEE CG&A has an article on benchmarking graphics
workstations.  From my limited knowledge of performance numbers, the
he graphics benchmark results seem reasonable.  However, the stated
results from the CPU benchmarks indicate that the SGI 4D/60 (MIPS
R2000/R2010 8MHz) is SLOWER than the old SGI 2400T (68020 16MHz) and
LESS than 1/4 the speed of a Sun-3/260 (68020 25MhZ)!

>From a graph comparing CPU performance in the article:

		Est. from graph		Reported	
Computer	% of Vax8600		Seconds

Vax 8600	100%			30
Sun 3/260	100%			30
Apollo DN580	30%			99
SGI 2400T	30%			64 (inconsistent, gives 46%)
Iris 4D/60	25%			118
Vax 11/750 VMS	20%			149
Vax 11/750 Unix  5%			489

The benchmark was a multivariate least-squares program.  Although the
article doesn't mention what language, compiler optimization level,
compiler version, level of FP precision, or the floating point
hardware that were used, the 4D/60 results seem very unlikely to me
regardless of what combination of these was used. I've benchmarked a
variety FP intensive C code (including applications using both 32bit
and 64bit FP).  The 4D/70 (MIPS R2000/R2010 12.5MHz) was 3-6 time
FASTER than a 3/260 w/FPA board.  Even derating the 4D/60 for a slower
clock speed of 8MHz, this is more than a factor of 10 off from their
results.

I don't think even RISC-bashers like Neal Nelson et al. could come up
with a benchmark that could generate this sort of discrepency.  As
someone who has had to spend (spelled w-a-s-t-e) lots of time running
benchmarks because some vendors' numbers can't be trusted, I think
that independent benchmark reports like this article are invaluable.
But it bothers the hell out of me when apparently misleading numbers
and incomplete descriptions like these get published in a widely
distributed publication.

It's a good thing I know that my numbers are the only right ones :-),
otherwise I might have acted on information like this.

Jim Helman
Department of Applied Physics			P.O. Box 10494
Stanford University				Stanford, CA 94309
(jim at thrush.stanford.edu) 			(415) 723-4940