Getting the most for a process.

[Jim Frost]

In article <12034 at cgl.ucsf.EDU> seibel at cgl.ucsf.edu (George Seibel) writes:
|In article <593 at cogent.UUCP> doug at cogent.UUCP (Doug Perlich) writes:
|>More exactly what other methods are there to get screaming performance out of
|>a UNIX machine.

|
|Even if you get 100% of the machine, you only go as fast as the machine
|will run your program.  Here's what to do if you *really* want to go fast:

Very valid techniques.  There are others that may work depending on
your application which can squeeze even more performance out of your
machine.

UNIX splits up the CPU amongst processes (and threads if your UNIX
supports them).  A single application can thus get more CPU at the
same priority if you can break the job up between multiple processes
or threads.  It's generally easier to do this with threads than with
separate processes, but you can do pretty well with separate processes
and shared memory for a lot of tasks -- particularly sequential
independent calculations (often found inside loops).

A simple shell script illustrates the principle:

	#! /bin/csh
	foreach i (/usr/*)
	  find $i -name foo
	end

versus:

	#! /bin/csh
	foreach i (/usr/*)
	  find $i -name foo &
	end

The latter will finish much faster (unless it thrashes the system),
but it has at least one problem -- output will become intermingled
unless you give each process its own output file.  The same sorts of
problems will have to be solved for a real application; see any
operating systems book for lots of solutions.

This method of parallelism is seen most often on multiprocessor
machines since processes will tend to execute on separate processors
and you get incredible throughput improvements.  On the Encore
Multimax this technique is used for grep and make, for instance.  The
technique still works on single processors -- obviously so since the
whole idea behind multitasking is to fully utilize the CPU (amongst
other resources), but usually not as well.

|>It seems to me the answer is no because then every programmer would try this
|>in his/her code and the system would halt.

Yes, this technique will definitely hurt the system if you run too
many parallel processes, but so will any technique that gives a single
application more than its "share" of CPU.  It's also a lot harder
because you have to coordinate processes so many people won't bother
unless they really need it or the system makes such a thing easy (I
have yet to see a system where it was particularly easy to parallelize
tasks effectively :-).

jim frost
software tool & die
madd at std.com