Unix Programming - memory leak?

Author

memory leak?

Aaron Walker

2004-01-23, 5:01 pm

As stated in other previous posts, I am attempting to write a http
server. I was testing it out, and opened up gnome system monitor to
look at the mem usage and noticed that each time I request a page, the
child processes' memory usage gradually grows. They all start out at
around 180k. After serving about 30 files or so (basically hitting
reload in the browser that many times), all the children were using
about 1.5MB each. Am I leaking memory?

Here is child_main() that each child process runs after the initial
forking at the start of the program.

static void child_main(int i, int listenfd, int addrlen)
{
char *url;
int clientfd;
socklen_t client_addrlen;
struct sockaddr_in client_addr;

if(verbose)
printf("Starting child process %d [%ld]\n", i + 1,
(long)getpid());

for(;;) {
client_addrlen = addrlen;
if((clientfd = accept(listenfd,
(struct sockaddr *)&client_addr,
&client_addrlen)) == -1)
{
if(errno == EINTR)
continue;
else {
fprintf(stderr, "accept: %s\n",
strerror(errno));
exit(1);
}

url = dissect_request(clientfd,
inet_ntoa(client_addr.sin_addr));
if(url) {
process_request(clientfd, url);
free(url);
}
}
close(clientfd);
}
}

I can't see how it could be leaking memory. dissect_request() calls
malloc() for url and then I free() it after processing the client's
request. process_request calls malloc() once for a temporary char
string, but then free()'s it before returning to child_main(). Once the
main server loop starts in each child process, it stays in that loop
until until the parent process recieves a signal, so I know if there was
a leak, it would have to be something inside that main loop.

any suggestions?

I appreciate all the help everyone has given me so far. I plan on
sticking around comp.unix.programmer for a while, and hopefully once I
am more knowledgable I can help others like I have been helped so far.

Once again, thanks.

Aaron

David Schwartz

2004-01-23, 5:01 pm

"Aaron Walker" <ka0ttic@REMOVETHIScfl.rr.com> wrote in message
news:ZTcyb.131869$86.1499733@twister.tampabay.rr.com...

quote:

> As stated in other previous posts, I am attempting to write a http
> server. I was testing it out, and opened up gnome system monitor to
> look at the mem usage and noticed that each time I request a page, the
> child processes' memory usage gradually grows. They all start out at
> around 180k. After serving about 30 files or so (basically hitting
> reload in the browser that many times), all the children were using
> about 1.5MB each. Am I leaking memory?

It sounds like you are.

quote:

> I can't see how it could be leaking memory. dissect_request() calls
> malloc() for url and then I free() it after processing the client's
> request. process_request calls malloc() once for a temporary char
> string, but then free()'s it before returning to child_main(). Once the
> main server loop starts in each child process, it stays in that loop
> until until the parent process recieves a signal, so I know if there was
> a leak, it would have to be something inside that main loop.
>
> any suggestions?

You say that all the children grew in memory? Perhaps the parent grew in
memory and each child started out with the parent's memory consumption.

We need more details.

DS

those who know me have no need of my name

2004-01-23, 5:01 pm

in comp.unix.programmer i read:

quote:

>As stated in other previous posts, I am attempting to write a http
>server. I was testing it out, and opened up gnome system monitor to
>look at the mem usage and noticed that each time I request a page, the
>child processes' memory usage gradually grows. They all start out at
>around 180k. After serving about 30 files or so (basically hitting
>reload in the browser that many times), all the children were using
>about 1.5MB each. Am I leaking memory?

could be. it's also possible that your memory allocator is retaining the
memory it has requested, the maximum being whatever was necessary for the
largest file served to that point.

--
a signature

David Schwartz

2004-01-23, 5:01 pm

"Aaron Walker" <ka0ttic@REMOVETHIScfl.rr.com> wrote in message
news:ZTcyb.131869$86.1499733@twister.tampabay.rr.com...

quote:

> As stated in other previous posts, I am attempting to write a http
> server. I was testing it out, and opened up gnome system monitor to
> look at the mem usage and noticed that each time I request a page, the
> child processes' memory usage gradually grows. They all start out at
> around 180k. After serving about 30 files or so (basically hitting
> reload in the browser that many times), all the children were using
> about 1.5MB each. Am I leaking memory?

It sounds like you are.

quote:

> I can't see how it could be leaking memory. dissect_request() calls
> malloc() for url and then I free() it after processing the client's
> request. process_request calls malloc() once for a temporary char
> string, but then free()'s it before returning to child_main(). Once the
> main server loop starts in each child process, it stays in that loop
> until until the parent process recieves a signal, so I know if there was
> a leak, it would have to be something inside that main loop.
>
> any suggestions?

You say that all the children grew in memory? Perhaps the parent grew in
memory and each child started out with the parent's memory consumption.

We need more details.

DS

those who know me have no need of my name

2004-01-23, 5:01 pm

in comp.unix.programmer i read:

quote:

>As stated in other previous posts, I am attempting to write a http
>server. I was testing it out, and opened up gnome system monitor to
>look at the mem usage and noticed that each time I request a page, the
>child processes' memory usage gradually grows. They all start out at
>around 180k. After serving about 30 files or so (basically hitting
>reload in the browser that many times), all the children were using
>about 1.5MB each. Am I leaking memory?

could be. it's also possible that your memory allocator is retaining the
memory it has requested, the maximum being whatever was necessary for the
largest file served to that point.

--
a signature

Aaron Walker

2004-01-23, 5:01 pm

David Schwartz wrote:

quote:

> You say that all the children grew in memory? Perhaps the parent grew in
> memory and each child started out with the parent's memory consumption.
>
> We need more details.
>
> DS
>
>

Ok, well I dug a little deeper, and tested it out on both of my systems
(freebsd and linux), and here is the output of ps for both systems, when
the program was first started and then after serving ~30 pages:

FreeBSD:
PID COMMAND PPID RSS VSZ
1617 ./lwhttpd -v 619 600 904
1618 ./lwhttpd -v 1617 572 904
1619 ./lwhttpd -v 1617 572 904
1620 ./lwhttpd -v 1617 572 904

1617 ./lwhttpd -v 619 600 904
1618 ./lwhttpd -v 1617 700 2192
1619 ./lwhttpd -v 1617 700 2192
1620 ./lwhttpd -v 1617 700 2192

Linux:
PID COMMAND PPID RSS VSZ
11191 ./lwhttpd -v 28539 424 1308
19010 ./lwhttpd -v 11191 428 1308
17638 ./lwhttpd -v 11191 428 1308
7897 ./lwhttpd -v 11191 428 1308

11191 ./lwhttpd -v 28539 424 1308
19010 ./lwhttpd -v 11191 440 1312
17638 ./lwhttpd -v 11191 440 1312
7897 ./lwhttpd -v 11191 440 1312

It looks fine on linux. There's barely a memory increase compared to
FreeBSD (which is what I was using when I posted the first time).

Aaron Walker

2004-01-23, 5:01 pm

David Schwartz wrote:

quote:

> You say that all the children grew in memory? Perhaps the parent grew in
> memory and each child started out with the parent's memory consumption.
>
> We need more details.
>
> DS
>
>

inf

2004-01-23, 5:01 pm

Aaron Walker wrote:

quote:

> As stated in other previous posts, I am attempting to write a http
> server. I was testing it out, and opened up gnome system monitor to
> look at the mem usage and noticed that each time I request a page, the
> child processes' memory usage gradually grows. They all start out at
> around 180k. After serving about 30 files or so (basically hitting
> reload in the browser that many times), all the children were using
> about 1.5MB each. Am I leaking memory?
>
> Here is child_main() that each child process runs after the initial
> forking at the start of the program.
>
> static void child_main(int i, int listenfd, int addrlen)
> {
> char *url;
> int clientfd;
> socklen_t client_addrlen;
> struct sockaddr_in client_addr;
>
> if(verbose)
> printf("Starting child process %d [%ld]\n", i + 1,
> (long)getpid());
>
> for(;;) {
> client_addrlen = addrlen;
> if((clientfd = accept(listenfd,
> (struct sockaddr *)&client_addr,
> &client_addrlen)) == -1)
> {
> if(errno == EINTR)
> continue;
> else {
> fprintf(stderr, "accept: %s\n",
> strerror(errno));
> exit(1);
> }
>
> url = dissect_request(clientfd,
> inet_ntoa(client_addr.sin_addr));
> if(url) {
> process_request(clientfd, url);
> free(url);
> }
> }
> close(clientfd);
> }
> }
>
> I can't see how it could be leaking memory. dissect_request() calls
> malloc() for url and then I free() it after processing the client's
> request. process_request calls malloc() once for a temporary char
> string, but then free()'s it before returning to child_main(). Once the
> main server loop starts in each child process, it stays in that loop
> until until the parent process recieves a signal, so I know if there was
> a leak, it would have to be something inside that main loop.
>
> any suggestions?
>
> I appreciate all the help everyone has given me so far. I plan on
> sticking around comp.unix.programmer for a while, and hopefully once I
> am more knowledgable I can help others like I have been helped so far.
>
> Once again, thanks.
>
> Aaron
>

a good way to do some hacking is to use strace and watch for calls to
brk() and see the allocation/deallocation scheme like someone else had
mentioned above. with a small experimental program you can learn alot
about how malloc works. we were discussing something slightly relevant
over here the other day:
http://forums.devshed.com/t101628/s.html

inf

2004-01-23, 5:01 pm

Aaron Walker wrote:

quote:

> As stated in other previous posts, I am attempting to write a http
> server. I was testing it out, and opened up gnome system monitor to
> look at the mem usage and noticed that each time I request a page, the
> child processes' memory usage gradually grows. They all start out at
> around 180k. After serving about 30 files or so (basically hitting
> reload in the browser that many times), all the children were using
> about 1.5MB each. Am I leaking memory?
>
> Here is child_main() that each child process runs after the initial
> forking at the start of the program.
>
> static void child_main(int i, int listenfd, int addrlen)
> {
> char *url;
> int clientfd;
> socklen_t client_addrlen;
> struct sockaddr_in client_addr;
>
> if(verbose)
> printf("Starting child process %d [%ld]\n", i + 1,
> (long)getpid());
>
> for(;;) {
> client_addrlen = addrlen;
> if((clientfd = accept(listenfd,
> (struct sockaddr *)&client_addr,
> &client_addrlen)) == -1)
> {
> if(errno == EINTR)
> continue;
> else {
> fprintf(stderr, "accept: %s\n",
> strerror(errno));
> exit(1);
> }
>
> url = dissect_request(clientfd,
> inet_ntoa(client_addr.sin_addr));
> if(url) {
> process_request(clientfd, url);
> free(url);
> }
> }
> close(clientfd);
> }
> }
>
> I can't see how it could be leaking memory. dissect_request() calls
> malloc() for url and then I free() it after processing the client's
> request. process_request calls malloc() once for a temporary char
> string, but then free()'s it before returning to child_main(). Once the
> main server loop starts in each child process, it stays in that loop
> until until the parent process recieves a signal, so I know if there was
> a leak, it would have to be something inside that main loop.
>
> any suggestions?
>
> I appreciate all the help everyone has given me so far. I plan on
> sticking around comp.unix.programmer for a while, and hopefully once I
> am more knowledgable I can help others like I have been helped so far.
>
> Once again, thanks.
>
> Aaron
>

Frank Pilhofer

2004-01-23, 5:01 pm

Aaron Walker <ka0ttic@REMOVETHIScfl.rr.com> wrote:

quote:

>
> I can't see how it could be leaking memory.
> any suggestions?
>

Check if your server keeps increasing in size. Write a shell script
to download 1000 pages (use wget). Note the size, then run your script
again.

Or just use valgrind (or Purify or your memory leak tool of choice).

Frank

--
Frank Pilhofer ........................................... fp@fpx.de
Life is what happens to you while you're busy making future plans.
- Alfred E. Neuman

Frank Pilhofer

2004-01-23, 5:01 pm

Aaron Walker <ka0ttic@REMOVETHIScfl.rr.com> wrote:

quote:

>
> I can't see how it could be leaking memory.
> any suggestions?
>

Bjorn Reese

2004-01-23, 5:01 pm

On Sun, 30 Nov 2003 22:04:37 +0000, Frank Pilhofer wrote:

quote:

> Or just use valgrind (or Purify or your memory leak tool of choice).

A list can be found at:

http://www.cs.colorado.edu/homes/zo...allocDebug.html

--
mail1dotstofanetdotdk

Bjorn Reese

2004-01-23, 5:01 pm

On Sun, 30 Nov 2003 22:04:37 +0000, Frank Pilhofer wrote:

quote:

> Or just use valgrind (or Purify or your memory leak tool of choice).

A list can be found at:

http://www.cs.colorado.edu/homes/zo...allocDebug.html

--
mail1dotstofanetdotdk