|
Home > Archive > Unix Programming > January 2004 > expanding memory usage, is that ok ?
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
expanding memory usage, is that ok ?
|
|
| kamran@uio.no 2004-01-23, 5:01 pm |
|
Hi
I have a read something similar to the following in the
faq but I humbly want to make sure that it is saying
what I think it is saying. Please forgive.
I have an application that depending on the size of the
incoming data reallocates memory for that data:
frame = (BYTE *)realloc(frame, FRAME_SIZE(&handles[0]));
These happen to be seismic data that could vary in amplitude,
hence the "FRAME_SIZE(&handles[0])" in the above statement
is changing all the time. Occasionally the data may contain
large events meaning the required size for "frame" expands
accordingly. (time bound packets, each 10 seconds of data).
Of course there is a limit how large the
amplitude of the event can be and accordingly how large
the size of the "handles" is, which is much much less than
it ever could become critical 
The size is usually 50-70kB but now and again it
can rise up to 300kB or more to then go back to the
normal size.
Still when a large data segment arrives the process monitor
report increased memory usage for the application and it
is always expanding and is not reduced when data gos back to
normal size.
I was wondering if this is ok though the memory is reported to
be expanding all the time, I mean that the memory is regained
by the operating system when other applications ask for it
or is it a potential hazardous situation, a real memory leak ?
Thanks in advance
Kamran
| |
| Jens.Toerring@physik.fu-berlin.de 2004-01-23, 5:01 pm |
| In comp.unix.questions kamran@uio.no wrote:quote:
> I have a read something similar to the following in the
> faq but I humbly want to make sure that it is saying
> what I think it is saying. Please forgive.
quote:
> I have an application that depending on the size of the
> incoming data reallocates memory for that data:
quote:
> frame = (BYTE *)realloc(frame, FRAME_SIZE(&handles[0]));
Two things here that you might want to change: First, unless
you plan to exit() immediately if realloc() returns NULL you
should store what 'frame' was before the call of realloc()
or you got yourself a memory leak. Second, drop the cast
of the return value of realloc() - it doesn't help in any
way and only will keep the compiler from complaining if you
should forget to include the header file with the declaration
of realloc(), <stdlib.h>.
And, BTW, '&handles[0]' is exactly the same as 'handles'.
quote:
> These happen to be seismic data that could vary in amplitude,
> hence the "FRAME_SIZE(&handles[0])" in the above statement
> is changing all the time. Occasionally the data may contain
> large events meaning the required size for "frame" expands
> accordingly. (time bound packets, each 10 seconds of data).
> Of course there is a limit how large the
> amplitude of the event can be and accordingly how large
> the size of the "handles" is, which is much much less than
> it ever could become critical
> The size is usually 50-70kB but now and again it
> can rise up to 300kB or more to then go back to the
> normal size.
> Still when a large data segment arrives the process monitor
> report increased memory usage for the application and it
> is always expanding and is not reduced when data gos back to
> normal size.
That's completely normal behaviour. The memory handling functions
keep their own pool of memory they got from the OS and usually
never give anything back to the OS until the program finishes.
So, instead of each time asking the OS when new memory is asked
for they first try to satisfy the request from the pool they
already got. No memory leaks involved and nothing to worry
about.
Regards, Jens
--
_ _____ _____
| ||_ _||_ _| Jens.Toerring@physik.fu-berlin.de
_ | | | | | |
| |_| | | | | | http://www.physik.fu-berlin.de/~toerring
\___/ens|_|homs|_|oerring
| |
| kamran@uio.no 2004-01-23, 5:01 pm |
|
Jens.Toerring@physik.fu-berlin.de wrote:quote:
> In comp.unix.questions kamran@uio.no wrote:
>
>
>
>
>
>
>
> Two things here that you might want to change: First, unless
> you plan to exit() immediately if realloc() returns NULL you
> should store what 'frame' was before the call of realloc()
> or you got yourself a memory leak. Second, drop the cast
> of the return value of realloc() - it doesn't help in any
> way and only will keep the compiler from complaining if you
> should forget to include the header file with the declaration
> of realloc(), <stdlib.h>.
>
> And, BTW, '&handles[0]' is exactly the same as 'handles'.
>
>
>
>
> That's completely normal behaviour. The memory handling functions
> keep their own pool of memory they got from the OS and usually
> never give anything back to the OS until the program finishes.
> So, instead of each time asking the OS when new memory is asked
> for they first try to satisfy the request from the pool they
> already got. No memory leaks involved and nothing to worry
> about.
> Regards, Jens
Thanking you a lot for the reply.
One thing about what you mention above, that: " ... usually
never give anything back to the OS until the program finishes",
the program is never to finish. It is an on-line program going on
forever, as it is the case with earth seismic activity
Still nothing to worry ?
Thank you again
Kamran
| |
| =?iso-8859-1?q?M=E5ns_Rullg=E5rd?= 2004-01-23, 5:01 pm |
| kamran@uio.no writes:
quote:
> Thanking you a lot for the reply.
> One thing about what you mention above, that: " ... usually
> never give anything back to the OS until the program finishes",
> the program is never to finish. It is an on-line program going on
> forever, as it is the case with earth seismic activity
> Still nothing to worry ?
You could use realloc only to expand the block, and never reduce the
size, i.e. the block would have the maximum size encountered. That
would eliminate the risk of fragmenting the memory and requiring more
from the OS.
--
Måns Rullgård
mru@kth.se
| |
| Chuck Dillon 2004-01-23, 5:01 pm |
| kamran@uio.no wrote:quote:
> Thanking you a lot for the reply.
> One thing about what you mention above, that: " ... usually
> never give anything back to the OS until the program finishes",
> the program is never to finish. It is an on-line program going on
> forever, as it is the case with earth seismic activity
> Still nothing to worry ?
Yes, still no worries. The memory footprint of the process will
reflect it's largest virtual memory use while running. So if you
monitor the change in a process's size over time you should see it
growing early on and then level off once it's encountered the full
range of inputs it's expected to come across. In your case it would
reflect the most intense seismic activity during the time the program
is running.
If you see the size growing consistently and have no explaination for
the growth then you are probably looking at a memory leak.
-- ced
--
Chuck Dillon
Senior Software Engineer
NimbleGen Systems Inc.
| |
| Mark R.Bannister 2004-01-23, 5:01 pm |
| kamran@uio.no wrote in message news:<3FAA29E8.707@uio.no>...quote:
> I have an application that depending on the size of the
> incoming data reallocates memory for that data:
>
> frame = (BYTE *)realloc(frame, FRAME_SIZE(&handles[0]));
<snip>quote:
> Still when a large data segment arrives the process monitor
> report increased memory usage for the application and it
> is always expanding and is not reduced when data gos back to
> normal size.
> I was wondering if this is ok though the memory is reported to
> be expanding all the time, I mean that the memory is regained
> by the operating system when other applications ask for it
> or is it a potential hazardous situation, a real memory leak ?
>
>
> Thanks in advance
>
> Kamran
Hi Kamran,
Don't worry, this isn't a memory leak. The peak memory allocated
remains available to the program on the heap, and will be re-allocated
again if there is another peak. Also, if the operating system gets
quite short of memory, it will steal some of these pages back if
they're not being used.
Best regards,
Mark.
| |
| kamran 2004-01-23, 5:01 pm |
| Thanks Mark
That was exactly the answer I was looking for,
that the memory is reclaimed by the operating system.
Now I can get my boss off my back.
Thanks again
Kamran
Mark R.Bannister wrote:
quote:
> kamran@uio.no wrote in message news:<3FAA29E8.707@uio.no>...
>
>
> <snip>
>
>
>
> Hi Kamran,
>
> Don't worry, this isn't a memory leak. The peak memory allocated
> remains available to the program on the heap, and will be re-allocated
> again if there is another peak. Also, if the operating system gets
> quite short of memory, it will steal some of these pages back if
> they're not being used.
>
> Best regards,
> Mark.
| |
| kamran 2004-01-23, 5:01 pm |
| Thanks Mark
That was exactly the answer I was looking for,
that the memory is reclaimed by the operating system.
Now I can get my boss off my back.
Thanks again
Kamran
Mark R.Bannister wrote:
quote:
> kamran@uio.no wrote in message news:<3FAA29E8.707@uio.no>...
>
>
>
> <snip>
>
>
>
>
> Hi Kamran,
>
> Don't worry, this isn't a memory leak. The peak memory allocated
> remains available to the program on the heap, and will be re-allocated
> again if there is another peak. Also, if the operating system gets
> quite short of memory, it will steal some of these pages back if
> they're not being used.
>
> Best regards,
> Mark.
| |
| Logan Shaw 2004-01-23, 5:01 pm |
| kamran wrote:quote:
> That was exactly the answer I was looking for,
> that the memory is reclaimed by the operating system.
> Now I can get my boss off my back.
Sadly, it's not quite that simple. The RAM will be reclaimed
by the operating system. But the virtual memory will not be,
in most cases. However, this typically doesn't matter much.
Your virtual memory size will almost always approach some bounds
(unless you have a legitimate memory leak), and because the
inactive RAM is reclaimed, all this costs you is swap space.
These days, disk space is very cheap, so using swap space is not
a big deal at all.
It would be possible to always return memory to the OS
when you're not using it, but in practice, your program will
run slower that way and it'll be a little harder to write.
And there is very little practical benefit.
- Logan
|
|
|
|
|