[ http://issues.apache.org/jira/brows...2356
765 ]

Graham Dumpleton commented on MODPYTHON-77:
-------------------------------------------

Just to make this problem even more complicated, due to how Mac OS X handles
 unloading of dynamically loaded modules, parts of the solution presented he
re so far may not actually work on that platform.

On Mac OS X, if one takes the initial basic solution of storing the first in
terpreter as the "main_interpreter" and nothing else, the original problem g
oes away, however, Apache will crash when "apachectl restart" is run.

The problem is that when a "restart" occurs, Apache doesn't actually kill th
e parent process. What it does is to unload any loaded Apache modules, such 
as "mod_python.so" and then it will be reload them again. This results in al
l global data within "mod_p
ython.so" being thrown away. The "python_init()" function of the mod_python.
so will be called again to reinitialise all the lost global data and start o
ver, but it fails in doing this.

The reason it fails is that although mod_python.so is unloaded, the dynamica
lly loaded Python framework isn't. The first consequence of this is that whe
n calling python_init() the subsequent time, the Python framework is already
 initialised. Due to a prev
ious workaround incorporated into python_init() specifically because of Mac 
OS X, this luckily doesn't stop us from at least trying to initialise mod_py
thon again.

In initialising mod_python though, because the previous global data is lost,
 we need to store the initial interpreter state against main_interpreter aga
in. To do this, "PyThreadState_Get()->interp" is used. For this to work thou
gh, it is necessary that th
e call to "PyEval_InitThreads()" just prior to that point in the code has ac
quired the global lock and set up a current thread state. This will only occ
ur though for the very first call to PyEval_InitThreads(). Because the Pytho
n framework isn't unloaded
and Python thus is already in an initialised state, the PyEval_InitThreads()
 call doesn't do anything. Because there is no valid active thread state, th
e call to PyThreadState_Get() fails.

One can see this when some extra debugging is added to mod_python. For Mac O
S X we get:

# apachectl start

[Fri Nov 04 20:50:17 2005] [notice] mod_python: Creating 8 session m
utexes based on 6 max processes and 25 max threads.
[Fri Nov 04 20:50:17 2005] [error] mp_initialized=0
[Fri Nov 04 20:50:17 2005] [error] Py_IsInitialized()=0
[Fri Nov 04 20:50:17 2005] [error] interpreters=0
[Fri Nov 04 20:50:18 2005] [error] save interpreter
[Fri Nov 04 20:50:18 2005] [error] pid=805
[Fri Nov 04 20:50:18 2005] [error] ppid=1
[Fri Nov 04 20:50:18 2005] [error] done save interpreter
[Fri Nov 04 20:50:18 2005] [error] get thread state
[Fri Nov 04 20:50:18 2005] [error] okay got thread state
[Fri Nov 04 20:50:18 2005] [error] released interpreter
[Fri Nov 04 20:50:18 2005] [notice] Apache/2.0.51 (Unix) mod_python/
3.2.4b Python/2.3 configured -- resuming normal operations

# apachectl restart

[Fri Nov 04 20:50:26 2005] [warn] child process 807 still did not ex
it, sending a SIGTERM
[Fri Nov 04 20:50:27 2005] [notice] SIGHUP received.  Attempting to 
restart
[Fri Nov 04 20:50:27 2005] [notice] mod_python: Creating 8 session m
utexes based on 2 max processes and 25 max threads.
[Fri Nov 04 20:50:27 2005] [error] mp_initialized=0
[Fri Nov 04 20:50:27 2005] [error] Py_IsInitialized()=1
[Fri Nov 04 20:50:27 2005] [error] interpreters=0
[Fri Nov 04 20:50:27 2005] [error] save interpreter
[Fri Nov 04 20:50:27 2005] [error] pid=805
[Fri Nov 04 20:50:27 2005] [error] ppid=1
Fatal Python error: PyThreadState_Get: no current thread
[Fri Nov 04 20:50:27 2005] [notice] seg fault or similar nasty error
 detected in the parent process

You can see how it is the same process ID, that Python is already marked as 
initailised, yet "interpreters" and the "initialized" flag which were global
 data in mod_python.so have gone back to being 0. Finally, the PyThreadState
_Get() function dies.

If one looks at Linux, one instead sees:

# apachectl start

[Fri Nov 04 05:22:46 2005] [notice] mod_python: Creating 8 session m
utexes based on 150 max processes and 0 max threads.
[Fri Nov 04 05:22:46 2005] [error] mp_initialized=0
[Fri Nov 04 05:22:46 2005] [error] Py_IsInitialized()=0
[Fri Nov 04 05:22:46 2005] [error] interpreters=0
[Fri Nov 04 05:22:46 2005] [error] save interpreter
[Fri Nov 04 05:22:46 2005] [error] pid=8757
[Fri Nov 04 05:22:46 2005] [error] ppid=1
[Fri Nov 04 05:22:46 2005] [error] done save interpreter
[Fri Nov 04 05:22:46 2005] [error] get thread state
[Fri Nov 04 05:22:46 2005] [error] okay got thread state
[Fri Nov 04 05:22:46 2005] [error] released interpreter
[Fri Nov 04 05:22:46 2005] [notice] Apache/2.0.55 (Unix) mod_python/
3.2.4b Python/2.3.5 configured -- resuming normal operations

# apachectl restart

[Fri Nov 04 05:22:55 2005] [notice] SIGHUP received.  Attempting to 
restart
[Fri Nov 04 05:22:55 2005] [notice] mod_python: Creating 8 session m
utexes based on 150 max processes and 0 max threads.
[Fri Nov 04 05:22:55 2005] [error] mp_initialized=0
[Fri Nov 04 05:22:55 2005] [error] Py_IsInitialized()=0
[Fri Nov 04 05:22:55 2005] [error] interpreters=0
[Fri Nov 04 05:22:55 2005] [error] save interpreter
[Fri Nov 04 05:22:55 2005] [error] pid=8757
[Fri Nov 04 05:22:55 2005] [error] ppid=1
[Fri Nov 04 05:22:55 2005] [error] done save interpreter
[Fri Nov 04 05:22:55 2005] [error] get thread state
[Fri Nov 04 05:22:55 2005] [error] okay got thread state
[Fri Nov 04 05:22:55 2005] [error] released interpreter
[Fri Nov 04 05:22:55 2005] [notice] Apache/2.0.55 (Unix) mod_python/
3.2.4b Python/2.3.5 configured -- resuming normal operations

Note how Python isn't initialised on second time through. This indicates tha
t any instance of Python in Apache is also being unloaded and thus everythin
g starts over from a totally clean slate.

Anyway, that be an additional problem on Mac OS X that will have to be dealt
 with in finding a solution. Since the issue is in the parent process, am go
ing to investigate whether the initial interpreter should only be stored as 
main_interpreter in the chi
ld process. Ie., in PythonChildInitHandler(). This might avoid this issue.

More reports later. :-)

> The multiple interpreter concept of mod_python is broken for Python extens
ion modules since Python 2.3
> --------------------------------------------------------------------------
----------------------------
>
>          Key: MODPYTHON-77
>          URL: http://issues.apache.org/jira/browse/MODPYTHON-77
>      Project: mod_python
>         Type: Bug
>   Components: core
>     Versions: 3.1.4
>  Environment: Python >= 2.3
>     Reporter: Boyan Boyadjiev
>  Attachments: diff.txt, diff2.txt, diff3.txt, gil_test.c, gilstate.tar.gz,
 mod_python.c, mod_python.c.diff, mod_python.h.diff, src.zip
>
> The multiple interpreter concept of mod_python is broken for Python extens
ion modules since Python 2.3 because of the PEP 311 (Simplified Global Inter
preter Lock Acquisition for Extensions):
> ...
> Limitations and Exclusions
>     This proposal identifies a solution for extension authors with
>     complex multi-threaded requirements, but that only require a
>     single "PyInterpreterState".  There is no attempt to cater for
>     extensions that require multiple interpreter states.  At the time
>     of writing, no extension has been identified that requires
>     multiple PyInterpreterStates, and indeed it is not clear if that
>     facility works correctly in Python itself.
> ...
> For mod_python this means, that complex Python extensions won't work any more with
 Python >= 2.3, because they are supposed to work only with the first interpreter st
ate initialized for the current process (a problem we experienced). The first interp
ret
er state is not used by mod_python after the python_init is called.
> One solution, which works fine for me, is to save the first interpreter st
ate into the "interpreters" dictionary in the function python_init (MAIN_INT
ERPRETER is used as a key):
> static int python_init(apr_pool_t *p, apr_pool_t *ptemp,
>                        apr_pool_t *plog, server_rec *s)
> {
>     ...
>     /* initialize global Python interpreter if necessary */
>     if (! Py_IsInitialized())
>     {
>         /* initialze the interpreter */
>         Py_Initialize();
> #ifdef WITH_THREAD
>         /* create and acquire the interpreter lock */
>         PyEval_InitThreads();
> #endif
>         /* create the obCallBack dictionary */
>         interpreters = PyDict_New();
>         if (! interpreters) {
>             ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, s,
>                          "python_init: PyDict_New() failed! No more memory
?");
>             exit(1);
>         }
>         {
>             /*
>             Workaround PEP 311 - Simplified Global Interpreter Lock Acquis
ition for Extensions
>             BEGIN
>             */
>             PyObject *p = 0;
>             interpreterdata * idata = (interpreterdata *)malloc(sizeof(int
erpreterdata));
>             PyThreadState* currentThreadState = PyThreadState_Get();
>             PyInterpreterState *istate = currentThreadState->interp;
>             idata->istate = istate;
>             /* obcallback will be created on first use */
>             idata->obcallback = NULL;
>             p = PyCObject_FromVoidPtr((void ) idata, NULL); /*p->refcout =
 1*/
>             PyDict_SetItemString(interpreters, MAIN_INTERPRETER, p); /*p->
refcout = 2*/
>             Py_DECREF(p); /*p->refcout = 1*/
>             /*
>             END
>             Workaround PEP 311 - Simplified Global Interpreter Lock Acquis
ition for Extensions
>             */
>         }
>         /* Release the thread state because we will never use
>          * the main interpreter, only sub interpreters created later. */
>         PyThreadState_Swap(NULL);
> #ifdef WITH_THREAD
>         /* release the lock; now other threads can run */
>         PyEval_ReleaseLock();
> #endif
>     }
>     return OK;
> }
> Another change I've made in the attached file is to Py_DECREF(p) in get_interprete
r, which will remove leaky reference to the PyCObject with the interpreter data. Thi
s was not a real problem, but now I see fewer leaks in BoundsChecker :-).

[ Post a follow-up to this message ]