Virtual memory behavior under SUNOS 4.0.3

[simon at corona.att.com]

Environment ->

Hardware :
    4/60 24 Meg ram w cgsix - diskless served by 4/260 w 48 Meg swap allocation

Software :
    Running X11R4, developing Xt / Openlook based graphics interfaces 
    typical application:
    size:
        text    data    bss     dec     hex          bss + data at run time
        540672  106496  0       647168  9e000        328 K
    ldd: (shared libaries being bound to)
        -lX11.4 => /usr/lib/libX11.so.4.2
        -lXmu.4 => /usr/lib/libXmu.so.4.0
        -lc.1 => /usr/lib/libc.so.1.3

I am noticing strange things on the sparkstation 1's we are developing on.
After a reboot vmstat reports a significant amount of free memory - 17896K
on 24 Meg system. After starting X11R4 development environment (2 xterms,
wm, xload) we are down to 12492K. After first make and application
execution 6196K.  Free memory dropped most precipitously during the link
edit (executible file size ~ 2 Meg).  After second make and application
execution 6296K. After third make of different version in different
directory and execution 5020K.  Where has 7Meg of free memory gone ??? No
shared memory is being used.  I can only think of the following
possibilities:

    Used pages are not immediately freed upon process death.
    Text pages are not considered free until memory pressure occurs. 
    Disk blocks are being locally cached in physical mem even when using NFS.

When multiple compiles and executions are done free memory monotonically
goes to between 256 and 1024K and paging activity increases. Incidently
all measurements where made when free memory was no longer changing.

Has anyone out there noticed this behavior ? I suspect other UNIX boxes do
not exhibit the same behavior. Of particular frustration to me is the
inability to get detailed system stats as to the # of vm segments active,
their virtual size, resident set size, disk buffering behavior (pmem used
by buffer ) etc.  I called the SUN info line but this question seemed to
be out of their league.

Marc Simon
ATT Bell Labs
simon at corona.ATT.COM 201-615-2535