vi core dumping on Sun 2 (fixed, but would like explanations)

[terryl at tekcrl.UUCP]

>Problem Description:
>	vi dumps core, repeatedly and reliably.

>How it started:
>Someone here had just ignored the old saying about not trying
>to fix that which is not broken,  and had just rebooted all our Sun 2/170's
>(in order to force fsck to check the disks) when the problem started
>(vi core dumping).

>NOTE the following:
>1) Binary hadn't been modified at all since pulled off the Sun release tape.
>2) Rcp-ed vi from another Sun over ethernet and did a ``cmp'' on the two
>   binarys and noted they were EXACTLY the same.
>3) Tried running the vi binary from the other Sun, noted it works FINE.
>4) Tried copying the vi binary to another file, works FINE.
>5) Get clever.  Notice vi has 6 links, make 5 links to the other Suns vi and
>   run it.  Works FINE.
>6) Get cleverer still.  Notice vi has sticky bit turned on, so turn on the
>   sticky bit of the other vi, try that.  Works FINE.
>7) Rebooted system, problem goes away.      <************* NOTE *************

     We saw this happen once on a VAX 11/780 with cp, of all programs!!! We
did exactly the same things that you did, with the exception of making the
multiple links(cp has no links). The problem disappeared on reboot. What we
think happened (and we were never able to prove it since the problem never
re-occurred), is that 4.2 (and 4.1, and 4.0) does not give up pages of pure-
text programs when the program exits, but marks them as pages of pure-text
programs in case if the program is run again soon, then the page does not
have to be read in again from disk. The theory is that the page somehow got
corrupted before its next use, but hadn't been reclaimed for some other use,
thus invalidating the incore copy of that text page. Unfortunately, when a
program dies and dumps core, only the data portion of the program is dumped,
and not the pure-text portion, so if one wanted to verify this theory, one
would have to get a core dump of PHYSICAL memory before rebooting and poke
around in there. We didn't think of this(and even if we did, we didn't feel
like poking around in PHYSICAL memory to test this theory).



					Terry Laskodi
					     of
					Tektronix