malloc problems revisited

[Paul Mielke]

In article <11421 at phoenix.Princeton.EDU>, markv at phoenix.Princeton.EDU
(Mark T Vandewettering) writes:
> 
> I know this has been hashed out in the newsgroups before, but I still
> have to reach a viable resolution for a program that I am working on.
> 
> This program allocates around 400K polygons, containing roughly 50 bytes
> each.  Yet, when drawn, the paging activity on the system goes through
> the roof.  Eventually, the program dies with by being unable to allocate
> any more pages.  We have 16 megabytes of memory on an SGI 240GTX, OS 3.1g.
> 

400k polys * 50 bytes/poly = 20 * 10**6 bytes = 20 megabytes

If your program actually needs to reference all 400k polys very frequently,
you're going to have a paging problem no matter what flavor of malloc
you use.  Your only hopes are 1) locality of reference or 2) buy more memory.

The problem you mention of the program eventually dying with an allocation
failure sounds like your program's virtual space is growing without bound.
Does your program continue to malloc memory throughout its life or does
it allocate the 400k polys once and for all?  If the virtual space of a program
grows without bound, malloc will eventually fail for lack of swap space in
which to store the expanded virtual space.  There are several ways to find
out what's going on:

ps -l and look at the SZ:RSS column.  This gives the number of pages (4kb)
of total virtual space and current resident space for each process.

'swap -l' will tell you how much of your swap space is currently allocated.
Watch the numbers as your program runs.

'df /debug' gives you more or less the same information as 'swap -l'. 
The 'avail'
and 'use' on 'df /debug' tell you the allocation of the sum total of
physical memory
and swap space.

> I have tried using -lmalloc to get the Berkeley one as well.  Does
> anyone have any good ideas as to how get around this problem?  Is there
> a problem in the library routines as well that prevents drawing of more
> than a fixed number of polygons.
> 
> As an aside, I am using the builtin lighting model.
> 
> Hope someone can help.
> 
> Mark (markv at cs.uoregon.edu)

Paul Mielke			paulm at sgi.com
Advanced Systems Division		(415) 962-3447
Silicon Graphics, Inc.