Heap memory usage

Florian Weingarten · External Since: Apr 24, 2009 Posts: 1

Hi everybody,

I would like to know how much heap memory my software is using.
I have been playing around with mallinfo() and malloc_stats()
and that seems to be what I want. However, the values they report
(arena) looked quite small (some megabytes), so I double checked
with valgrind, which reports a more likely value (about a gigabyte).

Where does that huge difference come from? I guess valgrind is
right and malloc is wrong. Is there another way to get (at runtime,
without slowing down my software too much) the current heap memory
usage? I tried getrusage() too, but that does not work at all.

Thanks in advance.

Flo

John Reiser · External Since: Apr 24, 2009 Posts: 3

> I would like to know how much heap memory my software is using.

That request is somewhat vague. Do you want
A) current value of sum(sizeof(malloc)) - sum(sizeof(free)))
B) maximum historical value of A)
C) current value of sum(sizeof(arena))
D) maximum historical value of C)
or something else? Beware of peaks and valleys in usage, fragmentation
of arenas, the difference between sbrk() and mmap(), etc.

> I have been playing around with mallinfo() and malloc_stats()
> and that seems to be what I want. However, the values they report
> (arena) looked quite small (some megabytes), so I double checked
> with valgrind, which reports a more likely value (about a gigabyte).

Note that the fields returned by mallinfo() are all int, which is 32
bits wide on most machines today. On x86_64, then mallinfo() has
aliasing problems. Please give specific details.

> I tried getrusage() too, but that does not work at all.

Please be specific. What were you trying to measure, what did
you observe (the values of which members of struct rusage),
and what did you expect?

--

Hadron · External Since: Dec 05, 2006 Posts: 495

John Reiser <jreiserfl.DeleteThis@comcast.net> writes:

>> I would like to know how much heap memory my software is using.
>
> That request is somewhat vague. Do you want
> A) current value of sum(sizeof(malloc)) - sum(sizeof(free)))
> B) maximum historical value of A)
> C) current value of sum(sizeof(arena))
> D) maximum historical value of C)
> or something else? Beware of peaks and valleys in usage, fragmentation
> of arenas, the difference between sbrk() and mmap(), etc.
>
>> I have been playing around with mallinfo() and malloc_stats()
>> and that seems to be what I want. However, the values they report
>> (arena) looked quite small (some megabytes), so I double checked
>> with valgrind, which reports a more likely value (about a gigabyte).
>
> Note that the fields returned by mallinfo() are all int, which is 32
> bits wide on most machines today. On x86_64, then mallinfo() has
> aliasing problems. Please give specific details.
>
>> I tried getrusage() too, but that does not work at all.
>
> Please be specific. What were you trying to measure, what did
> you observe (the values of which members of struct rusage),
> and what did you expect?

What was not specific about "how much heap memory his program is using"?

Clearly that would be at any single point in time he determines.

Peaks and valleys are nothing to with the measurement at any one point
in time.

--
In view of all the deadly computer viruses that have been spreading
lately, Weekend Update would like to remind you: when you link up to
another computer, you're linking up to every computer that that
computer has ever linked up to. — Dennis Miller

John Reiser · External Since: Apr 24, 2009 Posts: 3

Hadron wrote:
> What was not specific about "how much heap memory his program is using"?
>
> Clearly that would be at any single point in time he determines.

The _when_ quantifier is not stated explicitly, and common usage of
"is using" is sloppy enough to cover both an instant and wider intervals
of time. Inserting the "redundant" quantifier adds so much clarity
that it is essential to effective communication.

> Peaks and valleys are nothing to with the measurement at any one point
> in time.

The original poster compared statistics from valgrind(memcheck) to info
retrieved by mallinfo(). valgrind(memcheck) reports a number of different
kinds of information. The default report at the end of execution, such as:
malloc/free: in use at exit: 0 bytes in 0 blocks.
malloc/free: 38 allocs, 38 frees, 3,403 bytes allocated.
is a summary ("totals") which does not reflect instantaneous usage, except
as an upper bound. In contrast, mallinfo() does give instantaneous usage.
Comparing a total to an instantaneous measurement (except as a bound)
further illustrates the lack of precision in the original question.

--

Florian Weingarten · External Since: Apr 25, 2009 Posts: 1

John Reiser <jreiserfl.DeleteThis@comcast.net> wrote:
> That request is somewhat vague. Do you want
> A) current value of sum(sizeof(malloc)) - sum(sizeof(free)))
> B) maximum historical value of A)
> C) current value of sum(sizeof(arena))
> D) maximum historical value of C)
> or something else? Beware of peaks and valleys in usage, fragmentation
> of arenas, the difference between sbrk() and mmap(), etc.

I think in my situation, it doesnt matter. I am actually interested in
peaks, but here it is the same I guess, because my program just
malloc()s more and more and nothing is free()t until the end.

What I would like to do is: At some point in the program (at the end, but before
I free()), print "Currently n bytes allocated which are not yet free()d".

> Note that the fields returned by mallinfo() are all int, which is 32
> bits wide on most machines today. On x86_64, then mallinfo() has
> aliasing problems. Please give specific details.

Yeah, but int should do up to 4 GiB, I am not coming close to that limit,
so there should be no overflows here.

>> I tried getrusage() too, but that does not work at all.
> Please be specific. What were you trying to measure, what did
> you observe (the values of which members of struct rusage),
> and what did you expect?

I tried to read that "max rss" field, which was just zero all the
time (I read somewhere, that this is common on most Linux systems,
since its somehow not implemented or something).

Thanks for your reply

Flo