Relational Database Responses ("holes" in files)

[John Bruner]

[I'm responding to this in UNIX-WIZARDS instead of INFO-UNIX because
I think it is more appropriate for that newsgroup/mailing-list.]

> "Holes" are a well-known - and even documented - feature (deliberate
> feature - sparse files can save lots of disk space) of UNIX ever since
> V6 (although they didn't work as well in V6 as in post-V6 systems - reading
> a "hole" in V6 caused the blocks to be allocated).

One severe misfeature of "holes" in V6 was that the blocks would be
allocated when the file was read EVEN IF THE FILESYSTEM WAS READONLY.
If the filesystem was not physically write-protected this could
corrupt it.

It turns out that the "POSSIBLE FILE SIZE ERROR" messages can be
useful in a bizarre case.  If a crash leaves a "hole" in a directory
(a condition which cannot occur normally, since disc blocks are
never released from pre-4.2BSD directories), "namei" will get confused
when trying to read the directory (I don't have the V7 source code
handy, but as I recall "bmap" returns -1, but "namei" does not
check for this because it "can't happen").  The device driver for
the disc may catch the reference to a negative block number (many
drivers check for a block number which is too large, but not
for a negative block number) or it may decode the negative block
number and try to seek to a non-existent cylinder.  In any event,
you can't read the directory beyond the missing block, and you
may even be rewarded with a crash.

This happened to us at Purdue/ECN a few years ago after a
particularly nasty crash on a V7 11/70.  We had to track down the
file size errors, find the directories which were involved,
and either unlink them or fix them the really hard way (adb).
This problem was present in all of the versions of UNIX that
I was aware of at the time except for V6 (because it automatically
filled in "holes").  At the time I posted a bug fix for
V7/4.[01]BSD/2.8BSD.  I'll look for it, but since it was so
long ago (and so far away) I'm not optimistic that I could find
it to post it again.

The 4.2BSD code is different.  I haven't looked at it extensively,
but Berkeley sent me a note after I reported the bug in 4.1BSD
saying that the problem would be fixed in 4.2.

I have no idea if it has been fixed in System V.
-- 
  John Bruner (S-1 Project, Lawrence Livermore National Laboratory)
  MILNET: jdb at mordor.ARPA [jdb at s1-c]	(415) 422-0758
  UUCP: ...!ucbvax!dual!mordor!jdb 	...!decvax!decwrl!mordor!jdb