|
Home > Archive > Apache Mod-Python > July 2006 > Need confirmation of memory leak using Apache 2.2.2.
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Need confirmation of memory leak using Apache 2.2.2.
|
|
| Graham Dumpleton 2006-07-31, 1:12 am |
| I am using Apache 2.2.2 and when using mod_python in a certain way, I am seeing
significant ongoing increases in memory use by Apache child processes.
Initially I thought I had stuffed up recent changes in mod_python 3.3 out of
subversion trunk that I had been making, but I went back to mod_python 3.2.10
and am seeing the same problem.
Can some one please run the following test and use "top" or some other means of
monitoring memory use to see if you can duplicate the problem. The test is really
quite simple actually.
# .htaccess
PythonFixupHandler handlers
AddHandler mod_python .py
PythonHandler handlers
# handlers.py
from mod_python import apache
def handler(req):
req.content_type = 'text/plain'
req.write('handler')
return apache.OK
def fixuphandler(req):
return apache.OK
The command I have been using to test is:
ab -c 1 -n 5000 http://localhost:8082/~grahamd/MODPYTHON-155/hello.py
What is strange is that if I have only the fixup handler or response handler
invoked and not both, then memory leak isn't present. It is only when both are
being invoked that it leaks memory.
This occurs for me on Mac OS X 10.4, Python 2.3.5 and Apache 2.2.2, with
either mod_python 3.2.10 or mod_python 3.3. I haven't yet tested with older
Apache 2.0.58 as yet as don't have it on my box at present, but will download
it and see if that makes a difference.
I guess what I am trying to working out is if this is an issue with Apache 2.2.2
or whether it is mod_python and problem has been there for a while. Thus if
people with both Apache 2.0.X and Apache 2.2.X could test it it would be great.
Any help appreciated.
Thanks.
Graham
| |
| Graham Dumpleton 2006-07-31, 7:12 am |
| I get it on Apache 2.0.59 as well. :-(
I will thus be interested to see what others get, as appears to be an existing
mod_python issue.
BTW, this is with worker MPM.
Graham
Graham Dumpleton wrote ..
> I am using Apache 2.2.2 and when using mod_python in a certain way, I am
> seeing
> significant ongoing increases in memory use by Apache child processes.
> Initially I thought I had stuffed up recent changes in mod_python 3.3 out
> of
> subversion trunk that I had been making, but I went back to mod_python
> 3.2.10
> and am seeing the same problem.
>
> Can some one please run the following test and use "top" or some other
> means of
> monitoring memory use to see if you can duplicate the problem. The test
> is really
> quite simple actually.
>
> # .htaccess
>
> PythonFixupHandler handlers
> AddHandler mod_python .py
> PythonHandler handlers
>
> # handlers.py
>
> from mod_python import apache
>
> def handler(req):
> req.content_type = 'text/plain'
> req.write('handler')
> return apache.OK
>
> def fixuphandler(req):
> return apache.OK
>
> The command I have been using to test is:
>
> ab -c 1 -n 5000 http://localhost:8082/~grahamd/MODPYTHON-155/hello.py
>
> What is strange is that if I have only the fixup handler or response handler
> invoked and not both, then memory leak isn't present. It is only when both
> are
> being invoked that it leaks memory.
>
> This occurs for me on Mac OS X 10.4, Python 2.3.5 and Apache 2.2.2, with
> either mod_python 3.2.10 or mod_python 3.3. I haven't yet tested with older
> Apache 2.0.58 as yet as don't have it on my box at present, but will download
> it and see if that makes a difference.
>
> I guess what I am trying to working out is if this is an issue with Apache
> 2.2.2
> or whether it is mod_python and problem has been there for a while. Thus
> if
> people with both Apache 2.0.X and Apache 2.2.X could test it it would be
> great.
>
> Any help appreciated.
>
> Thanks.
>
> Graham
| |
| Graham Dumpleton 2006-07-31, 7:12 am |
| The good news is that this isn't a problem introduced with mod_python
3.2.10
so doesn't necessarily have to stop that being released.
The bad news though is that is is also in mod_python 3.2.8, so it has
been
there for some time.
The question now is whether anyone else sees it on other platforms or
whether
it is something specific to my machine.
Graham
On 31/07/2006, at 4:53 PM, Graham Dumpleton wrote:
[vbcol=seagreen]
> I get it on Apache 2.0.59 as well. :-(
>
> I will thus be interested to see what others get, as appears to be
> an existing
> mod_python issue.
>
> BTW, this is with worker MPM.
>
> Graham
>
> Graham Dumpleton wrote ..
| |
| Graham Dumpleton 2006-07-31, 7:12 am |
| Okay, found the source of the memory leak. The problem goes right back
to 3.1.4 which also has the problem when tested.
The problem code is in python_handler() in 'src/mod_python.c'.
Specifically
the code does:
if (!hle) {
/* create a handler list object from dynamically registered
handlers */
request_obj->hlo = (hlistobject *)MpHList_FromHLEntry
(dynhle, 1);
}
else {
/* create a handler list object */
request_obj->hlo = (hlistobject *)MpHList_FromHLEntry(hle, 1);
/* add dynamically registered handlers, if any */
if (dynhle) {
MpHList_Append(request_obj->hlo, dynhle);
}
}
Problem is that request_obj->hlo can already be set by a prior phase's
handler and by simply assigning to request_obj->hlo you get a memory
leak as it is a Python object and it isn't being decref'd.
Thus, before this 'if' statement, it would appear that the following
should
be inserted.
if (request_obj->hlo)
Py_DECREF(request_obj->hlo);
or:
Py_XDECREF(request_obj->hlo);
Still need to do some more checking, but adding that seems to get rid of
the problem.
Now what do we do about 3.2.10? Given that this thing leaks really badly
when triggered shows that no one must be using multiple handler phases
at the same time, so may be safe to still release 3.2.10 and we fix
it in next
backport release and 3.3.
Comments.
Graham
On 31/07/2006, at 7:24 PM, Graham Dumpleton wrote:
[vbcol=seagreen]
> The good news is that this isn't a problem introduced with
> mod_python 3.2.10
> so doesn't necessarily have to stop that being released.
>
> The bad news though is that is is also in mod_python 3.2.8, so it
> has been
> there for some time.
>
> The question now is whether anyone else sees it on other platforms
> or whether
> it is something specific to my machine.
>
> Graham
>
> On 31/07/2006, at 4:53 PM, Graham Dumpleton wrote:
>
| |
| Max Bowsher 2006-07-31, 7:12 am |
| Graham Dumpleton wrote:
> Okay, found the source of the memory leak. The problem goes right back
> to 3.1.4 which also has the problem when tested.
...
> Now what do we do about 3.2.10? Given that this thing leaks really badly
> when triggered shows that no one must be using multiple handler phases
> at the same time, so may be safe to still release 3.2.10 and we fix it
> in next backport release and 3.3.
Since this is a confirmed non-regression, especially one which has
existed for a long time, I don't think it makes sense to call off 3.2.10
- especially not at this late stage when all that is still needed is
official rubber-stamping of the release.
There are two very important new features coming in this release:
* Apache 2.2 support
* don't die in configure when using a recent bash
Max.
| |
| Jim Gallacher 2006-07-31, 7:12 am |
| Here is further confirmation that it leaks like crazy for:
mod_python 3.2.10, Linux Ubuntu 6.06, Apache 2.0.55 (mpm-worker), Python
2.4.3
Jim
Graham Dumpleton wrote:
> I get it on Apache 2.0.59 as well. :-(
>
> I will thus be interested to see what others get, as appears to be an existing
> mod_python issue.
>
> BTW, this is with worker MPM.
>
> Graham
>
> Graham Dumpleton wrote ..
>
| |
| Jim Gallacher 2006-07-31, 7:12 am |
| Max Bowsher wrote:
> Graham Dumpleton wrote:
> ...
>
> Since this is a confirmed non-regression, especially one which has
> existed for a long time, I don't think it makes sense to call off 3.2.10
> - especially not at this late stage when all that is still needed is
> official rubber-stamping of the release.
>
> There are two very important new features coming in this release:
> * Apache 2.2 support
> * don't die in configure when using a recent bash
>
Agreed. Hopefully Grisha will have a chance to sign and upload the
3.2.10 tarball in the near future.
Jim
|
|
|
|
|