IOCALL results and problems

[Dan Ts'o]

In article <354 at ncr-sd.UUCP> stubbs at ncr-sd.UUCP (0000-Jan Stubbs) writes:
>
>	IOCALL, A UNIX SYSTEM PERFORMANCE BENCHMARK
>The results so far are below. Thanks everybody.
>Send your results to me directly. The benchmark is a "C" program
>which measures Unix kernel performance. 
>
>
>/*This benchmark tests speed of Unix system call interface
>  and speed of cpu doing common Unix io system calls. */
>
>char buf[512];
>int fd,count,i,j;
>
>main()
>{
> fd = creat("/tmp/testfile",0777);
> close(fd);
>  fd = open("/tmp/testfile",2);
>  unlink("/tmp/testfile");
>for (i=0;i<=1000;i++) {
>  lseek(fd,0L,0);		/* add this line! */
>  count = write(fd,buf,500);
>  lseek(fd,0L,0);		/* second argument must be long */
>
>  for (j=0;j<=3;j++) 
>  	count = read(fd,buf,100);
>  }
>}

	Well I don't want to flame too much. Just a few comments.

	Basically, I find it difficult to take this benchmark and the presented
results too seriously.

	- I have trouble understanding the point of the benchmark program. It
just seems bizarre. For 1000 times, it writes 500 bytes at the beginning of the
file and reads 400 of them back, 100 at a time. Because of the buffer cache,
this whole routine just does user/kernel buffer copies, back and forth. If the
performance of the system call interface and user/kernel memory copies is the
what is trying to be measured, then the results may be okay, although strangely
obtained. I don't believe it measures much else in the way of kernel
performance, or system performance.  Its not even something a normal user can
relate to, such as "copying files on a X is twice as fast as Y".

	- It is obviously a single point measurement. It can tell you very
little about how particular applications or the system in general will run.

	- The numbers are way to small to interpret with any substantial
significance (i.e. you should run the benchmark with say 10000, rather than 1000
in the the loop). The difference between the various VAX 11/750 times are,
for example 7.2 to 9.4 . I could be convince there is significance there, but...

	- That a Radio Shack 16A performs 25% better than a VAX 11/750 is cute
but little practical interest (read ridiculous, a benchmark that tells me that
is probably not going to be very useful, are we really to think that an
Amdahl 470/V8 is only 12% faster than a VAX8600, that a Pyramid is slower than
a VAX 11/780).