brk's zero-fill behavior on VAXen

[Jan Edler]

The NYU Ultracomputer prototype has been running in various stages of
hardware and software development since 1982, and very early on we
decided that the kernel (based mostly on v7) would not clear newly
allocated memory.  It has not been shown to cause any problems.  I
don't recall ever having trouble with this when porting old programs
(dereferencing null pointers and byte-ordering problems are much more
pervasive).

We thought we had a bug once that was being caused by this, so we put
an optional feature in the kernel to spread a given garbage-pattern on
newly allocated memory, and spent some more time tracking down the
problem, only to find that it was caused by something else (hardware).
We kept the optional garbage-spreading feature, although it hasn't been
used in a long time.

This does not alter the fact that an uninitialized-variable bug in a
program can be nondeterministic, but having the kernel set newly
allocated memory to some value doesn't completely eliminate the problem
(e.g. if the uninitialized variable is on the stack, it can still have
an arbitrary value).

Of course, not setting newly allocated memory to some value is clearly
a weakness from a security point of view.

Jan Edler
New York University
edler at nyu
cmcl2!edler