Hi,

could someone explain me how malloc() (particularly doug lea's) deals
with programs who calls sbrk on their own.

What happens to the "Wilderness" chunk?

Thanks.

<rootmaniac@gmail.com> wrote in message 
news:1141591823.410268.232530@u72g2000cwu.googlegroups.com...

> could someone explain me how malloc() (particularly doug lea's) deals
> with programs who calls sbrk on their own.
>
> What happens to the "Wilderness" chunk?

    Nothing. The 'malloc' library didn't get that chunk from the system, so 
it doesn't do anything with it.

    DS

On 2006-03-05, David Schwartz <davids@webmaster.com> wrote:
>
> <rootmaniac@gmail.com> wrote in message 
> news:1141591823.410268.232530@u72g2000cwu.googlegroups.com...
>
>> could someone explain me how malloc() (particularly doug lea's) deals
>> with programs who calls sbrk on their own.
>>
>> What happens to the "Wilderness" chunk?
>
>     Nothing. The 'malloc' library didn't get that chunk from the
> system, so it doesn't do anything with it.

Is it guaranteed that malloc can cope with having brk() called
elsewhere? SUSv2 seems to say that the behavior is unspecified.

"Jordan Abel" <random832@gmail.com> wrote in message 
news:slrne0mm5h.1a75.random832@random.yi.org...
> On 2006-03-05, David Schwartz <davids@webmaster.com> wrote:
>>
>> <rootmaniac@gmail.com> wrote in message
>> news:1141591823.410268.232530@u72g2000cwu.googlegroups.com...
>>
>>> could someone explain me how malloc() (particularly doug lea's) deals
>>> with programs who calls sbrk on their own.
>>>
>>> What happens to the "Wilderness" chunk?
>>
>>     Nothing. The 'malloc' library didn't get that chunk from the
>> system, so it doesn't do anything with it.
>
> Is it guaranteed that malloc can cope with having brk() called
> elsewhere? SUSv2 seems to say that the behavior is unspecified.

    It is not guaranteed in general; however, I don't know of any 'malloc' 
library that has a problem with it. Most of the modern ones don't even use 
'brk' anymore, they use 'mmap'.

    DS

They use mmap in case of big chunks (> 1024 I think).

Seb

<rootmaniac@gmail.com> wrote in message 
news:1141640944.135199.134680@v46g2000cwv.googlegroups.com...

> They use mmap in case of big chunks (> 1024 I think).
>
> Seb

    They return the mmap'd block in the case of big chunks. For smaller 
chunks, they grab a big chunk with 'mmap' and split it up. In neither case 
do they use brk or sbrk to get the blocks. (For most modern 'malloc' 
implementations.)

    DS

David Schwartz wrote:
> <rootmaniac@gmail.com> wrote in message
> news:1141640944.135199.134680@v46g2000cwv.googlegroups.com...
>
> > They use mmap in case of big chunks (> 1024 I think).
> >
> > Seb
>
>     They return the mmap'd block in the case of big chunks. For smaller
> chunks, they grab a big chunk with 'mmap' and split it up. In neither case
> do they use brk or sbrk to get the blocks. (For most modern 'malloc'
> implementations.)
>
>     DS

Sorry I didn't know this.
I was refering to doug lea's malloc wich uses sbrk.

Seb

"David Schwartz" <davids@webmaster.com> writes:

>     They return the mmap'd block in the case of big chunks. For smaller 
> chunks, they grab a big chunk with 'mmap' and split it up. In neither case 
> do they use brk or sbrk to get the blocks. (For most modern 'malloc' 
> implementations.)

Your definition of "most modern malloc implementations" is apparently
different from mine.

On *all* systems I work on (recent Linux, Solaris, AIX), the
*default* malloc implementation in libc (not libumem, etc.), does
*not* use mmap for small blocks.

For example, on Solaris 10:
$ echo "int main() { int i; for (i = 0; i < 1000; ++i) malloc(100); return 0; }" > junk.c &&
  cc -g junk.c && truss ./a.out
....
getrlimit(RLIMIT_STACK, 0xFFBFF1A0)             = 0
getpid()                                        = 24497 [24496]
setustack(0xFF3A2088)
brk(0x00020D70)                                 = 0
brk(0x00022D70)                                 = 0
brk(0x00022D70)                                 = 0
brk(0x00024D70)                                 = 0
brk(0x00024D70)                                 = 0
...
brk(0x0003CD70)                                 = 0
_exit(0)


Cheers,
-- 
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

"Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message 
news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...

> On *all* systems I work on (recent Linux, Solaris, AIX), the
> *default* malloc implementation in libc (not libumem, etc.), does
> *not* use mmap for small blocks.

    Wow. All I can say is "wow". I wonder why they do this. I can think of 
no technical reason.

    DS

On 2006-03-06, David Schwartz <davids@webmaster.com> wrote:
>
> "Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message 
> news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...
>
>> On *all* systems I work on (recent Linux, Solaris, AIX), the
>> *default* malloc implementation in libc (not libumem, etc.), does
>> *not* use mmap for small blocks.
>
>     Wow. All I can say is "wow". I wonder why they do this. I can think of 
> no technical reason.
>
>     DS

FreeBSD uses brk, but the implementation contains code that appears to
be intended to deal with non-malloc-related use of brk by the
application. [the macro MALLOC_NOT_MINE and related code]

"David Schwartz" <davids@webmaster.com> writes:


>"Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message 
>news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...

>> On *all* systems I work on (recent Linux, Solaris, AIX), the
>> *default* malloc implementation in libc (not libumem, etc.), does
>> *not* use mmap for small blocks.

>    Wow. All I can say is "wow". I wonder why they do this. I can think of 
>no technical reason.

Other than "performance sucks when you use mmap, munmap a lot"?


(There was a recent thread were someone complained that glibc performed
very poorly with his application and that the kernel was spending 30% of
the CPU time in zeroing pages.  The reason was that it use mmap/munmap
for allocations over a certain size)

Casper
-- 
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Casper H.S. Dik <Casper.Dik@Sun.COM> writes:

> (There was a recent thread were someone complained that glibc performed
> very poorly with his application and that the kernel was spending 30% of
> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
> for allocations over a certain size)

Are you sure it was a thread and not this article:
http://blogs.sun.com/roller/page/ahl?entry=dtrace_for_linux

Cheers,
-- 
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

Paul Pluzhnikov <ppluzhnikov-nsp@charter.net> writes:

>Casper H.S. Dik <Casper.Dik@Sun.COM> writes:

>> (There was a recent thread were someone complained that glibc performed
>> very poorly with his application and that the kernel was spending 30% of
>> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
>> for allocations over a certain size)

>Are you sure it was a thread and not this article:
>http://blogs.sun.com/roller/page/ahl?entry=dtrace_for_linux


Yes, I was sure it was a thread.

Casper
-- 
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

"Casper H.S. Dik" <Casper.Dik@Sun.COM> wrote in message
news:440cd184$0$11076$e4fe514c@news.xs4all.nl...
> (There was a recent thread were someone complained that glibc performed
> very poorly with his application and that the kernel was spending 30% of
> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
> for allocations over a certain size)

Sounds like the reason was more specifically that the application frequently
allocated and deallocated large blocks, and the deallocator could and did
return the memory to the OS.

Alex

"Alex Fraser" <me@privacy.net> writes:

>"Casper H.S. Dik" <Casper.Dik@Sun.COM> wrote in message
>news:440cd184$0$11076$e4fe514c@news.xs4all.nl...
>> (There was a recent thread were someone complained that glibc performed
>> very poorly with his application and that the kernel was spending 30% of
>> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
>> for allocations over a certain size)

>Sounds like the reason was more specifically that the application frequently
>allocated and deallocated large blocks, and the deallocator could and did
>return the memory to the OS.

Yes; and for that usage pattern such behaviour is suboptimal (many
allocators have a worst case scenario and this one application
was hurt particularly bad).

Casper
-- 
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

On 6 Mar 2006 21:39:44 GMT, Jordan Abel <random832@gmail.com> wrote:
> On 2006-03-06, David Schwartz <davids@webmaster.com> wrote:
>>
>> "Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message
>> news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...
>>
>>> On *all* systems I work on (recent Linux, Solaris, AIX), the
>>> *default* malloc implementation in libc (not libumem, etc.), does
>>> *not* use mmap for small blocks.
>>
>> Wow. All I can say is "wow". I wonder why they do this. I can think
>> of no technical reason.
>>
>>     DS
>
> FreeBSD uses brk, but the implementation contains code that appears to
> be intended to deal with non-malloc-related use of brk by the
> application. [the macro MALLOC_NOT_MINE and related code]

FWIW,

malloc() in CURRENT has been rewritten by Jason Evans to use mmap().
The runtime linker-loader uses mmap() too, so some of the mmap() calls
below are not allocations, but that's ok:

% $ echo "int main() { int i; for (i = 0; i < 1000; ++i) malloc(100); return 0; }" > junk.c && \
%   cc -g junk.c && ktrace ./a.out
% $ kdump -f ktrace.out | grep brk
% $ kdump -f ktrace.out | egrep -e 'NAMI|mmap'
%   2938 ktrace   NAMI  "./a.out"
%   2938 ktrace   NAMI  "/libexec/ld-elf.so.1"
%   2938 a.out    CALL  mmap(0,0x1e60,0x3,0x1000,0xffffffff,0,0)
%   2938 a.out    RET   mmap 5398528/0x800526000
%   2938 a.out    CALL  mmap(0,0x8000,0x3,0x1002,0xffffffff,0,0)
%   2938 a.out    RET   mmap 5398528/0x800526000
%   2938 a.out    NAMI  "/etc/libmap.conf"
%   2938 a.out    NAMI  "/var/run/ld-elf.so.hints"
%   2938 a.out    NAMI  "/lib/libc.so.6"
%   2938 a.out    NAMI  "/lib/libc.so.6"
%   2938 a.out    CALL  mmap(0,0x20b000,0x5,0x20002,0x3,0,0)
%   2938 a.out    RET   mmap 6496256/0x800632000
%   2938 a.out    CALL  mmap(0x80080d000,0x1a000,0x3,0x12,0x3,0,0xdb000)
%   2938 a.out    RET   mmap 8441856/0x80080d000
%   2938 a.out    CALL  mmap(0x800827000,0x16000,0x3,0x1012,0xffffffff,0,0)
%   2938 a.out    RET   mmap 8548352/0x800827000
%   2938 a.out    CALL  mmap(0,0x110,0x3,0x1000,0xffffffff,0,0)
%   2938 a.out    RET   mmap 5431296/0x80052e000
%   2938 a.out    CALL  mmap(0,0xb100,0x3,0x1000,0xffffffff,0,0)
%   2938 a.out    RET   mmap 5431296/0x80052e000
%   2938 a.out    NAMI  "/etc/malloc.conf"
%   2938 a.out    CALL  mmap(0,0x2000000,0x3,0x1002,0xffffffff,0x800000000,0)
%   2938 a.out    RET   mmap 8638464/0x80083d000
%   2938 a.out    CALL  mmap(0,0x2000000,0x3,0x1002,0xffffffff,0,0)
%   2938 a.out    RET   mmap 33554432/0x802000000
% $

The next major release of FreeBSD (7.0-RELEASE) will use this allocator
by default.

- Giorgos

Casper H.S. Dik wrote:
> Other than "performance sucks when you use mmap, munmap a lot"?
> 
> 
> (There was a recent thread were someone complained that glibc performed
> very poorly with his application and that the kernel was spending 30% of
> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
> for allocations over a certain size)

A bit of a tangent but I hope people don't mind. The point of this 
subthread, and the blog entry pointed at below, seems to be that 
munmap() can be expensive. From other readings it seems this is 
particularly true on multiprocessor systems. But does it apply at all to 
a *read-only* mmap? I can't find anything in the mmap/munmap 
documentation which discusses this.

Thanks,
HT

On 2006-03-07, Giorgos Keramidas <keramida@ceid.upatras.gr> wrote:
> On 6 Mar 2006 21:39:44 GMT, Jordan Abel <random832@gmail.com> wrote:
>> On 2006-03-06, David Schwartz <davids@webmaster.com> wrote:
>>>
>>> "Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message
>>> news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...
>>>
>>>> On *all* systems I work on (recent Linux, Solaris, AIX), the
>>>> *default* malloc implementation in libc (not libumem, etc.), does
>>>> *not* use mmap for small blocks.
>>>
>>> Wow. All I can say is "wow". I wonder why they do this. I can think
>>> of no technical reason.
>>>
>>>     DS
>>
>> FreeBSD uses brk, but the implementation contains code that appears to
>> be intended to deal with non-malloc-related use of brk by the
>> application. [the macro MALLOC_NOT_MINE and related code]
>
> FWIW,
>
> malloc() in CURRENT has been rewritten by Jason Evans to use mmap().
> The runtime linker-loader uses mmap() too, so some of the mmap() calls
> below are not allocations, but that's ok:
>
> % $ echo "int main() { int i; for (i = 0; i < 1000; ++i) malloc(100); return 0; }" > junk.c && \
> %   cc -g junk.c && ktrace ./a.out
> % $ kdump -f ktrace.out | grep brk
> % $ kdump -f ktrace.out | egrep -e 'NAMI|mmap'
> %   2938 ktrace   NAMI  "./a.out"
> %   2938 ktrace   NAMI  "/libexec/ld-elf.so.1"
> %   2938 a.out    CALL  mmap(0,0x1e60,0x3,0x1000,0xffffffff,0,0)
> %   2938 a.out    RET   mmap 5398528/0x800526000
> %   2938 a.out    CALL  mmap(0,0x8000,0x3,0x1002,0xffffffff,0,0)
> %   2938 a.out    RET   mmap 5398528/0x800526000
> %   2938 a.out    NAMI  "/etc/libmap.conf"
> %   2938 a.out    NAMI  "/var/run/ld-elf.so.hints"
> %   2938 a.out    NAMI  "/lib/libc.so.6"
> %   2938 a.out    NAMI  "/lib/libc.so.6"
> %   2938 a.out    CALL  mmap(0,0x20b000,0x5,0x20002,0x3,0,0)
> %   2938 a.out    RET   mmap 6496256/0x800632000
> %   2938 a.out    CALL  mmap(0x80080d000,0x1a000,0x3,0x12,0x3,0,0xdb000)
> %   2938 a.out    RET   mmap 8441856/0x80080d000
> %   2938 a.out    CALL  mmap(0x800827000,0x16000,0x3,0x1012,0xffffffff,0,0)
> %   2938 a.out    RET   mmap 8548352/0x800827000
> %   2938 a.out    CALL  mmap(0,0x110,0x3,0x1000,0xffffffff,0,0)
> %   2938 a.out    RET   mmap 5431296/0x80052e000
> %   2938 a.out    CALL  mmap(0,0xb100,0x3,0x1000,0xffffffff,0,0)
> %   2938 a.out    RET   mmap 5431296/0x80052e000
> %   2938 a.out    NAMI  "/etc/malloc.conf"
> %   2938 a.out    CALL  mmap(0,0x2000000,0x3,0x1002,0xffffffff,0x800000000,0)
> %   2938 a.out    RET   mmap 8638464/0x80083d000
> %   2938 a.out    CALL  mmap(0,0x2000000,0x3,0x1002,0xffffffff,0,0)
> %   2938 a.out    RET   mmap 33554432/0x802000000
> % $
>
> The next major release of FreeBSD (7.0-RELEASE) will use this allocator
> by default.

Will there be a flag [in /etc/malloc.conf or MALLOC_OPTIONS] to change
this behavior back to the original?

"Casper H.S. Dik" <Casper.Dik@Sun.COM> wrote in message 
news:440cd184$0$11076$e4fe514c@news.xs4all.nl...

> "David Schwartz" <davids@webmaster.com> writes:

>>"Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message
>>news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...

>>> On *all* systems I work on (recent Linux, Solaris, AIX), the
>>> *default* malloc implementation in libc (not libumem, etc.), does
>>> *not* use mmap for small blocks.

>>    Wow. All I can say is "wow". I wonder why they do this. I can think of
>>no technical reason.

> Other than "performance sucks when you use mmap, munmap a lot"?

    You only *need* to call 'mmap' when your total address size grows. You 
never *need* to call 'munmap'. Any application that calls mmap/munmap a lot 
is using a badly designed allocator.

> (There was a recent thread were someone complained that glibc performed
> very poorly with his application and that the kernel was spending 30% of
> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
> for allocations over a certain size)

    And presumably not reusing the mappings.

    DS

On 2006-03-07, David Schwartz <davids@webmaster.com> wrote:
>
> "Casper H.S. Dik" <Casper.Dik@Sun.COM> wrote in message 
> news:440cd184$0$11076$e4fe514c@news.xs4all.nl...
>
>> "David Schwartz" <davids@webmaster.com> writes:
>
>>>"Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> wrote in message
>>>news:m3oe0jtwr5.fsf@somewhere.in.california.localhost...
>
>>>> On *all* systems I work on (recent Linux, Solaris, AIX), the
>>>> *default* malloc implementation in libc (not libumem, etc.), does
>>>> *not* use mmap for small blocks.
>
>>>    Wow. All I can say is "wow". I wonder why they do this. I can think of
>>>no technical reason.
>
>> Other than "performance sucks when you use mmap, munmap a lot"?
>
>     You only *need* to call 'mmap' when your total address size grows. You 
> never *need* to call 'munmap'. Any application that calls mmap/munmap a lot 
> is using a badly designed allocator.
>
>> (There was a recent thread were someone complained that glibc performed
>> very poorly with his application and that the kernel was spending 30% of
>> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
>> for allocations over a certain size)
>
>     And presumably not reusing the mappings.
>
>     DS

It can be useful to return freed allocations to the system so that your
memory usage is not monotonically increasing. Some malloc
implementations will even "brk backwards" when [rarely] they can.

"Jordan Abel" <random832@gmail.com> wrote in message 
news:slrne0rsik.2jr4.random832@random.yi.org...

> It can be useful to return freed allocations to the system so that your
> memory usage is not monotonically increasing. Some malloc
> implementations will even "brk backwards" when [rarely] they can.

    Sure, but it's boneheaded to do so when you just allocated from the 
system a fraction of a second ago. You never want to get into the situations 
where a tight loop allocating and freeing the same number of bytes results 
in a mmap/munmap cycle. If an allocator does this, then it's not suitable 
for this case.

    The company I work for has a house allocator that we use largely because 
it ensures consistent behavior on all platforms we support. What it does is, 
in any case when it might return memory to the system, check if it allocated 
memory from the system in the past second. If so, it keeps the memory in a 
pool. On its next defragment pass, if it hasn't allocated memory from the 
system recently, it will return blocks whose freeing was deferred.

    DS

On 2006-03-07, David Schwartz <davids@webmaster.com> wrote:
>
> "Jordan Abel" <random832@gmail.com> wrote in message 
> news:slrne0rsik.2jr4.random832@random.yi.org...
>
>> It can be useful to return freed allocations to the system so that your
>> memory usage is not monotonically increasing. Some malloc
>> implementations will even "brk backwards" when [rarely] they can.
>
>     Sure, but it's boneheaded to do so when you just allocated from the 
> system a fraction of a second ago. You never want to get into the situations 
> where a tight loop allocating and freeing the same number of bytes results 
> in a mmap/munmap cycle. If an allocator does this, then it's not suitable 
> for this case.
>
>     The company I work for has a house allocator that we use largely because 
> it ensures consistent behavior on all platforms we support. What it does is, 
> in any case when it might return memory to the system, check if it allocated 
> memory from the system in the past second. If so, it keeps the memory in a 
> pool. On its next defragment pass, if it hasn't allocated memory from the 
> system recently, it will return blocks whose freeing was deferred.

When does it make defragment passes? Unpredictable "background" stuff
happening at an arbitrary point can screw up algorithm complexity
calculations.

"Jordan Abel" <random832@gmail.com> wrote in message 
news:slrne0ru07.2jr4.random832@random.yi.org...

> When does it make defragment passes?

    Either on a schedule or when asked to. There's also an API to 
increment/decrement a "defragment disable" counter, though we've never found 
any use for it other than in benchmarking.

> Unpredictable "background" stuff
> happening at an arbitrary point can screw up algorithm complexity
> calculations.

    All the applications we've used it on are network server applications --  
databases, web servers, message relays, mail relays, and so on. It has an 
option to defragment as it goes, but it's generally a bad trade-off. A 
thread might call 'malloc' or 'free' while holding a critical lock or while 
at an elevated priority, so you don't want to take any more time in those 
calls than you have to. For what we do, deferring some of the hard work to a 
service thread is superior.

    DS

David Schwartz wrote:
> <rootmaniac@gmail.com> wrote in message
> news:1141640944.135199.134680@v46g2000cwv.googlegroups.com...
>
> > They use mmap in case of big chunks (> 1024 I think).
> >
> > Seb
>
>     They return the mmap'd block in the case of big chunks. For smaller
> chunks, they grab a big chunk with 'mmap' and split it up. In neither case
> do they use brk or sbrk to get the blocks. (For most modern 'malloc'
> implementations.)

This is not true for glibc.
http://groups.google.com/group/comp.unix.programmer/msg/b2c8fa643ac574df

Casper H.S. Dik wrote:

[]

> (There was a recent thread were someone complained that glibc performed
> very poorly with his application and that the kernel was spending 30% of
> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
> for allocations over a certain size)

The kernel can not avoid zeroing out pages after they were mapped to
the process virtual address space because the page frames might contain
sensitive data from another process.

On 7 Mar 2006 18:51:06 GMT, Jordan Abel <random832@gmail.com> wrote:
> On 2006-03-07, Giorgos Keramidas <keramida@ceid.upatras.gr> wrote:
>>> FreeBSD uses brk, but the implementation contains code that appears to
>>> be intended to deal with non-malloc-related use of brk by the
>>> application. [the macro MALLOC_NOT_MINE and related code]
>>
>> FWIW,
>>
>> malloc() in CURRENT has been rewritten by Jason Evans to use mmap().
>> [...]
>> The next major release of FreeBSD (7.0-RELEASE) will use this allocator
>> by default.
>
> Will there be a flag [in /etc/malloc.conf or MALLOC_OPTIONS] to change
> this behavior back to the original?

I don't think so.  At least, that's my understanding from all the work
Jason has put into optimizing the new malloc() to make sure that it
performs better for SMP systems and doesn't create a huge overhead for
single-processor systems at the same time.

- Giorgos

Giorgos Keramidas <keramida@ceid.upatras.gr> writes:
> Jordan Abel <random832@gmail.com> writes:
> > David Schwartz <davids@webmaster.com> writes:
> > > "Paul Pluzhnikov" <ppluzhnikov-nsp@charter.net> writes:
> > > > On *all* systems I work on (recent Linux, Solaris, AIX), the
> > > > *default* malloc implementation in libc (not libumem, etc.), does
> > > > *not* use mmap for small blocks.
> > > Wow. All I can say is "wow". I wonder why they do this. I can think
> > > of no technical reason.
> > FreeBSD uses brk, but the implementation contains code that appears to
> > be intended to deal with non-malloc-related use of brk by the
> > application. [the macro MALLOC_NOT_MINE and related code]
> malloc() in CURRENT has been rewritten by Jason Evans to use mmap().

Not on all platforms; i386, arm and powerpc still use brk().

Also, note that different allocators use mmap() in different ways.
FreeBSD's jemalloc uses mmap() (on some platforms) to allocate arenas
from which requests are satisfied, while glibc's malloc uses mmap() to
satisfy large requests directly, and brk() to allocate the arena from
which small requests are satisfied.

DES
-- 
Dag-Erling Sm�rgrav - des@des.no

"Maxim Yegorushkin" <maxim.yegorushkin@gmail.com> writes:


>Casper H.S. Dik wrote:

>> (There was a recent thread were someone complained that glibc performed
>> very poorly with his application and that the kernel was spending 30% of
>> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
>> for allocations over a certain size)

>The kernel can not avoid zeroing out pages after they were mapped to
>the process virtual address space because the page frames might contain
>sensitive data from another process.

Of course it cannot.  But for this application it would have been better
not to munmap() but just keep the mappings around (or use brk()).
The application is free not to zero the memory unless it absolutely has to.

Casper
-- 
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

On Mon, 06 Mar 2006 18:18:13 -0800, Paul Pluzhnikov wrote:

> Casper H.S. Dik <Casper.Dik@Sun.COM> writes:
> 
>> (There was a recent thread were someone complained that glibc performed
>> very poorly with his application and that the kernel was spending 30% of
>> the CPU time in zeroing pages.  The reason was that it use mmap/munmap
>> for allocations over a certain size)
> 
> Are you sure it was a thread and not this article:
> http://blogs.sun.com/roller/page/ahl?entry=dtrace_for_linux

 This is the thread Casper was talking about:

http://sourceware.org/ml/libc-alpha/2006-03/msg00033.html

-- 
James Antill -- james@and.org
http://www.and.org/and-httpd