huge memory allocation

[John F Haugh II]

In article <18016 at milton.u.washington.edu> dona at pooky.ee.washington.edu (Don Allen) writes:
>I am trying to run an application process which requires an allocation
>of about 150M of memory, but am unable to do so.  The largest amount
>of memory I have been able to get a single process to successfully
>acquire (using calloc or malloc, with either one or multiple calls)
>is about 33M; any larger request has NULL returned.  However, I can
>launch several applications simultaneously each of which acquire
>33M; so, it doesn't appear to be a limit on my account.

You are running up against one of two problems.  The most likely
one is that the data space ulimit for your account is probably
32MB or so (possibly 64MB ...).  This can prevent you from making
requests smaller than 128MB.  Raising your data limit high enough
will allow malloc() requests up to 128MB.

As for the 128MB limit, it is caused by a combination of two things.
First, the S/6000 has a 256MB segment size limit.  Each segment is
limited to 256MB, so you can't have a data segment larger than that.
What aggrevates that is that the malloc() on the S/6000 comes from
BSD and uses "power of 2" allocation, thus a 30MB request becomes
a 32MB request, and a 34MB request becomes a 64MB request.  You can
figure out what happens to that 150MB request - except that your
stack, data, and bss are already sharing the segment, so the malloc()
call has to fail because of the 256MB segment limit.

DISCLAIMER: I speak for myself only.
-- 
John F. Haugh II      |      I've Been Moved     |    MaBellNet: (512) 838-4340
SneakerNet: 809/1D064 |          AGAIN !         |      VNET: LCCB386 at AUSVMQ
BangNet: ..!cs.utexas.edu!ibmchs!auschs!snowball.austin.ibm.com!jfh (e-i-e-i-o)