|
Home > Archive > Unix Programming > May 2005 > Increased memory usage when a daemon is killed.
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Increased memory usage when a daemon is killed.
|
|
| Bharath 2005-05-03, 6:00 pm |
| Hello All,
I am experiencing a weird problem and was wondering if some
of you have a any clue about it.
I use a library (written in C++ but I don't have the source)
for starting a couple of daemon programs. These daemons fork
off processes when new connections come in. Usually, there
are around 10 processes running for each of these daemons.
Linux "top" command reports a system memory usage of 200MB
for both these when they are running.
When I restart these daemons by "kill"ing them, the memory
usage goes up to 600MB for about a minutes or so and this
causes a lot of problems (our system is a swapless system).
The excess memory usage during kill is directly proportional
to the number of processes running at that time. If there
are only 5 processes (for each daemon), the memory usage
goes up to 400MB when the daemons are killed.
Note that after that time, the usage drops down to 80MB.
Any clue why this is happening. The Kernel should reclaim
the memory resources as soon as the process is killed, right.
The guys who provided the library say that their C++ destructors
do not cause any excess memory usage.
Any clues??
Appreciate your help.
-- BK
| |
| phil_gg04@treefic.com 2005-05-03, 8:49 pm |
| > I use a library (written in C++ but I don't have the source)
> for starting a couple of daemon programs. These daemons fork
> off processes when new connections come in.
>
> When I restart these daemons by "kill"ing them, the memory
> usage goes up to 600MB for about a minutes or so and this
> causes a lot of problems (our system is a swapless system).
What signal do you send? If you "kill -9" then no signal handlers (and
definitely no C++ destructors) will run - do you still see the
unexpected memory use in that case?
I suggest doing a "strace -p" on the process as you kill it and see if
there are any clues in its output.
--Phil.
| |
| Bharath 2005-05-03, 8:49 pm |
| Phil,
Thanks for the response. A little clarification to my post.
I don't use "kill -9". But if I do, there is no problem at
all. Everything is very clean.
These process use some shared libraries -- so the following
mechanism is used.
When a daemon is to be killed, the parent sends a SIGTERM
to all the child processes it spawned. Each child does
something (essentially a function call to the 3rd party library..
and I don't know what they do in it -- and that was the first
culprit!) and then marks itself as "ready to die" in some
structure. The parent then kills all the child processes explicitly.
I like your strace idea, but that's specific to a process, right.
It doesn't capture the events from the child processes, isn't it.
I wanted to ensure that there is nothing strange in the OS
behavior.
| |
| David Schwartz 2005-05-04, 2:52 am |
|
"Bharath" <cbkumar@gmail.com> wrote in message
news:1115161165.399531.243820@z14g2000cwz.googlegroups.com...
> When I restart these daemons by "kill"ing them, the memory
> usage goes up to 600MB for about a minutes or so and this
> causes a lot of problems (our system is a swapless system).
You will have to explain to us what this means. What *exactly* are you
seeing?
> Any clue why this is happening. The Kernel should reclaim
> the memory resources as soon as the process is killed, right.
> The guys who provided the library say that their C++ destructors
> do not cause any excess memory usage.
It may simply be due to the lack of swap and overcommit being off. If a
50MB process forks, the kernel has to assume that each copy might modify
every single byte of memory it owns, resulting in 100MB of usage. If it
forks 5 copies, it must assume that each copy will modify every page it
owns.
DS
| |
|
| > I like your strace idea, but that's specific to a process, right.
> It doesn't capture the events from the child processes, isn't it.
strace can be used to follow forks,in which case it records the calls of
all the child processess.
strace -ff -o <output file> <program> ( You can also catch a running
process).
This obove options create different output files for every child process
including the parent process.
~
| |
| phil_gg04@treefic.com 2005-05-04, 7:50 am |
| > I like your strace idea, but that's specific to a process, right.
> It doesn't capture the events from the child processes, isn't it.
You can use it in various different ways; it can certainly follow
forks. I think I'd start the system up, establish some connections,
and then find the pid for just one of the spawned processes. Then
strace -p just that one, and look for clues. If that shows nothing
interesting, I'd strace just the "master" process.
--Phil.
|
|
|
|
|