Yep, this is getting a little hairy, but nothing we couldn't handle :-)

I did a little more research. Basically, this started with Graham's patch
that addressed a problem with modules being reimported (or something).
From Graham's message:

>
> The basic problem revolves around the Python dictionary used to hold the
> set of interpreters. The code in mod_python.c is trying to use the
> Python GIL to provide exclusive access to that dictionary and any
> subsequent creation of an interpreter.
>
> The only catch is that in creating a new interpreter, the Python core
> is, in someway I don't understand, swapping thread states at some point
> which is allowing other threads to acquire the GIL.
>

So what Graham's patch does is create an APR lock (interpreters_lock) and
wrap all the access to the dictionary with calls to apr_mutex_lock/unlock.

I think the _real_ way to address this issue is to first find what is the
problem with using the Python GIL to serialize access to the interpreters
dictionary. Is this a Python bug, or are we not understanding GIL and
using it improperly?

BUT, given that the above question may be complicated to answer, and that
Graham's patch resolves the issue, another thought:

If the APR lock works, what is the point of using the GIL in addition?
Should we just use the APR-based lock alone? I.e., where we had (after
Graham's patch):

#ifdef WITH_THREAD
 apr_thread_mutex_lock(interpreters_lock)
;
PyEval_AcquireLock();
#endif

we would use:

#ifdef APR_HAS_THREAD
 apr_thread_mutex_lock(interpreters_lock)
;
#endif

_without_ a call to PyEval_AcquireLock() at all.

It should compile OK, and on platforms where APR has no thread support,
like you said, it's not an issue since no separate interpreters run in one
process at the same time.

Grisha

On Mon, 12 Sep 2005, Nicolas Lehuen wrote:

> Duh, this is becoming difficult 
>
> I was thinking that if APR_HAS_THREADS was 0, then Apache was forcibly ran
> in prefork mode, so there was no need for thread safety at all, given the
> fact that mod_python would only run one interpreter thread. So if
> WITH_THREAD was not defined, ORAPR_HAS_THREADS was 0, then we would not ne
ed
> any thread safety code. Hence the definition of
> MOD_PYTHON_WITH_THREAD_SUPPORT.
>
> You're right in writing that a user could launch a new thread in Python,
> provided that WITH_THREAD is defined, even if APR_HAS_THREADS==0. However,
> having a look at the parts of mod_python.c where the thread safety was put
> in, I think we can safely say that those parts are only called by mod_pyth
on
> (through python_handler, python_cleanup etc who call get_interpreter). Tho
se
> parts are therefore always called in the same thread (if APR_HAS_THREADS==
0,
> that is) and there is no need for thread synchronization to be done (no
> shared data between the main thread and the other user threads, no need to
> release the GIL etc.).
>
> BUT, I could be very, very wrong here, and your idea of reverting to a
> conservative "shield Python threading calls with WITH_THREAD and apr
> threading code with APR_HAS_THREADS" is way more attractive to my tired mi
nd
> right now. So if you want I can revert all this
> MOD_PYTHON_WITH_THREAD_SUPPORT hack.
>
> Regards,
> Nicolas
>
> 2005/9/12, Gregory (Grisha) Trubetskoy <grisha@apache.org>: 
>

[ Post a follow-up to this message ]