unexplained cores after compiles

[was-John McMillan]

In article <610 at icus.islp.ny.us> lenny at icus.islp.ny.us (Lenny Tropiano) writes:
>	...
>The disadvantages to delayed write is that the user issuing the "write" system
>call is never sure when the data finally is written to the disk/media.
>Issuing a "sync" system call or using the "sync" command will assure that
>all dirty buffers are written to disk.

1)	The previous posting I made was correct regarding the existence
	of a window of vulnerability after the context-switch-out of
	a shared-memory process.  It was incorrect when it suggested
	the principle OTHER alternative was pilot error.

2)	Lenny is mostly correct in what he said, but his anthropomorphic,
	hope-filled kernel doesn't seem, to me, a very clear explanation
	of THE PROBLEM.  And "disadvantages to the delayed write" is
	an oblique way of stating that if you abuse a concept, it fails.
	
The problem: TWO forms of file-access are being used.

	A)	The LOADER [ld(1)] is using BIO [block I/O]
		to create the files.  This involves acquiring
		the disk space from the free-list, recording
		this in the INODE for the file, and writing the
		file image to disk.

	B)	The EXEC [sys1.c:getxfile()] uses two approaches
		to acquiring program image: 1) demand-loaded (PAGEIN)
		and 2) read-at-exec.  The commonly-used SHARED-TEXT
		images are demand-loaded.  (Read-at-exec uses
		BIO, presumably doesn't fail, and is ignored below.)

	A clash between BIO and PAGEIN occurs because the latter
	uses the former's BMAP [indices of file blocks] without
	using the former's cache-checking.

Specifically:
	BIO-accesses test if a desired block is INCORE -- is in
	a cached buffer -- before touching the disk.
	PAGEIN-accesses, however, presume the file is on disk
	-- saving the 4 cache searches / page
		[ 4 * 1KB Logical Block == 4KB Page ]
	which would encounter a HIT 'so rarely as to be ignorable'.

	[ 'so rarely as to be ignorable' == The Reported #$@%^ problem. ]

	This is, to me, an abuse of the BIO -- but with a reward/price
	ratio that is acceptable.  It should ONLY affect just-compiled
	codes so production software should be reliable UNLESS they
	are doing compile/exec's -- quite a rarity.

There are no inexpensive hooks permitting a test of WHEN the inode
was written: if there were, that time could be compared to the ROOTDEV's
last s_time and a flush (bio.c:bflush()) and a sleep() -- or incore()'s
-- might be used to synchronize the two mechanisms, or at least drop the
incidence of problems another few orders of magnitude [base 10].
	1) Inode 'times' are maintained on disk, but NOT in the
		inode table;
	2) Inode 'times' are slightly unreliable given their under-mining
		by TIME(1,2), CPIO/TAR, and dead-batteries.
			
If I wasn't so strange to begin with, I'd think it was odd that
I've never noticed this in 4 years of living with the s4/7300/3b1/#@$%!
			- - - - - - - - - - - -
Perhaps the LD(1) software should have included a SYNC(2) before exiting.

Note, however, SYNC-ing is a probabilistic work-around:
	SYNC(2) is not guaranteed to be COMPLETE -- just started -- upon return.

(In that vein: perhaps LD DOES PERFORM A SYNC!   -- but I doubt it.)
			- - - - - - - - - - - -
>It's very rare.  I would have trouble producing the error.

There are numerous reasons why attempts at FORCING this problem
to occur may fail:
	1) An SMGR sync(2) may have slip through;
	2) a flush may have been coerced by other buffer needs;
	3) re-compiling and CP-ing into the target directory tends
		to use the same physical-blocks as the previous
		image: depending on the location and amount
		code-changes, the Paged-In blocks from the previous
		version may be the same as those of the about-to-be-
		written new version.
	4) Bad karma, und so weiter...
			- - - - - - - - - - - -
>							  I had to stick
>"sync" system calls into my B-tree routines at work at strategic points to
>assure that the data is written out to the disk, hopefully not destroying
>the performance of the B-tree routines.

Huh?
If you are sticking with BIO, you do not need sync's for data accesses.

If you are sticking with RAW IO, you do not need sync's for data accesses.

If you are mixing the two, you need serious help -- much more than
a SYNC will provide -- unless it's filled with interesting chemicals !^}
			- - - - - - - - - - - -
>... but there I was able to tweek the NAUTOUP tunable parameter to decrease
>the time between the system will automatically update the dirty buffers.

Huh?
Again, there seems to be an allusion to some other problem than I
think we're discussing above.  Increasing NAUTOUP is a TUNING issue
-- it is unlikely that anyone need touch this [not that I give a flying
core dump if you DO].  Those dirty ol' buffers are just fine as they are.

If there's some problem beyond JUST-COMPILED codes, let's see it raised
in detail: until then, it sounds like pilot error.
			- - - - - - - - - - - -

Hmmm... the above has really been boring.  Apologies.

jc mcmillan	-- att!mtunb!jcm	-- speaking for himself, only
		-- save an electron this week: use 'r', not 'F', if possible.