Debian Developers - More dh

Author

More dh_python questions

Manoj Srivastava

2006-07-29, 1:24 am

Hi,

Digging into the current dh_python, I have come across
behaviour that I can't explain, related to the handling of the -V
flag. Take the case of private pure Python modules. Suppose I have
two packages, python-foo and python-bar. In python-foo, I call
dh_python -V X.Y, which means that the .py files are meant to be
handled by a specific version of python, namely pythonX.Y. Suppose,
in python-bar, I instead set XS-Python-Version to X.Y, which means
only pythonX.Y is supported.

First question: is XS-Python-Version a mandatory field under
policy? It should be, as we can see below, but currently Python
policy does not say so.

I posit that in either case it has been specified that only
version X.Y of Python is supported -- but the behaviour of dh_python
is different.

,----
| Package: python-foo
| Depends: pythonX.Y
| XB-Python-Version: X.Y
| Provides: pythonX.Y-foo
`----

,----
| Package: python-bar
| XB-Python-Version: X.Y
| Provides: pythonX.Y-bar
`----

Why are we not getting a Depends line in python-bar even
though XS-Python-Version states only one version of Python is
supported? If we had python-baz, an extension module, which has
XS-Python-Version: X.Y , we would get:

,----
| Package: python-baz
| XB-Python-Version: X.Y
| Provides: pythonX.Y-baz
| Depends: Python (>= X.Y), Python (<< X.Y+1)
`----

which makes sense. In other words, for private extension
modules, we ensure that only one version of Python is supported, and
no other, but we don't do so for private pure Python modules

Analysis of code follows.

Take chunk One. See how either $deps is modified, or
$verdeps{$usepython} is set, depending on whether or not -V flag was
used?
----------------------------------------------------------------------
291 if (/\.py$/ and $private_pydirs_regex and $fn =~ m/(?:$private_pydirs_regex)/) {
292 # Private Python module
293 verbose_print("Found private module: $fn");
294 my $dir;
295 foreach $dir (@dirs) {
296 if ($fn =~ m/\Q$dir\E/) {
297 $dir =~ s/^$tmp//;
298 verbose_print("Memorizing dir to byte-compile: $dir");
299 $private_dirs_list{"$dir"} = 1;
300 }
301 }
302 if ($dh{V_FLAG_SET}) {
303 $verdeps{$usepython} |= PY_PRIVATE_MODULE;
304 } else {
305 $deps |= PY_PRIVATE_MODULE;
306 }
307 } elsif (/\.so$/ and $private_sodirs_regex and $fn =~ m/(?:$private_sodirs_regex)/) {
----------------------------------------------------------------------

Next up: we add to depends for pure Python modules, but only if
-V had been called above.
----------------------------------------------------------------------
374 # Common dependency handling
375 foreach my $pyver (keys %verdeps) {
...
380 # Always add pythonX.Y dependency if some private modules are
381 # byte-compiled with it (or if extensions are
382 # byte-compiled with it)
383 if ($verdeps{$pyver} & (PY_PRIVATE_MODULE|SO_PRIVATE_MODULE)) {
384 addsubstvar($package, "python:Depends", $pyver);
385 }
386 }
----------------------------------------------------------------------

If we had not used -V, however, we get this: We set the
variable versions_field, either to current, or the contents of
XS-Python-Version. In the case of bar, we get X.Y.
----------------------------------------------------------------------
424 # Private modules
425 if ($deps & PY_PRIVATE_MODULE) {
426 # Package with private modules can only support one version at a time
427 # (even if the modules can be automatically byte-compiled for any new
428 # Python version).
429 unless ($versions_field) {
430 # Unless min/max are the same we put "current"
431 if ($min_version and ($min_version eq $max_version)) {
432 $versions_field = $min_version;
433 } else {
434 $versions_field = "current";
435 }
436 }
437 }
----------------------------------------------------------------------

What is done with that variable?
----------------------------------------------------------------------
445 if (scalar keys %pyversions_found) {
....
471 } else {
472 # Still try to describe the versions that the package support
473 $versions_field = $python_header unless $versions_field;
474 $versions_field = "all" unless $versions_field;
475 addsubstvar($package, "python:Versions", $versions_field);
476
477 # Generate provides for all Python versions supported
478 if ($package =~ /^python-/) {
479 foreach (grep { $officially_supported{$_} } @versions) {
480 my $virtual = $package;
481 $virtual =~ s/^python-/$python$_-/;
482 addsubstvar($package, "python:Provides", $virtual);
483 }
484 }
485 }
----------------------------------------------------------------------
Hmm, python_header is initialized from XS-Python-Version, which,
if it were mandatory, would mean that we would get X.Y in either
case. So, we get a XB-Python-Version, and we get a Provides, in
either case. But if XS-Python-Version is missing from python-foo, we
would get a XB-Python-Version: all and no provides, even though we
told dh_python that only one version of Python can run our .py files.

manoj
--
Trying to break out of the Tempter's control, one's mind writhes to
and fro, like a fish pulled from its watery home onto dry ground. 34
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Manoj Srivastava

2006-07-30, 7:20 pm

Hi,

I have a growing suspicion that dh_python does not do the
right thing for private pure Python modules in the presence of
XS-Python-version

This is how dh_python behaves:
*** PRIVATE PURE MODULE:
If there is a .py file, and it is in a private dir. only one
version is supported, even though the private module can be
recompiled for any new version.
**** Add dir to $private_dirs_list{"$dir"} (for recompilation)
**** add to the deps var if no -V option is given.
***** If versions_field is unset (so precedence goes to extensions),
****** if $min_version and min_version=max_version (1 version supported)
$versions_field = $min_version
****** else $versions_field = current ?????
**** or set $verdeps{X.Y} if only version X.Y is supported (-V)
***** Each entry is in the verdeps hash gets added to depends
**** No pyversions_found.
****** Set Versions to $versions_field

So, if I have a package with a private pure Python module, and
I have set XS-Python-Version to suggest that a ubset of python
versions are supported, and I have not used -V (since more than one
python version is to be supported).

What happens?
Case A:
I have only one version in XS-Python-Version (say, 2.4)
,----
| Package: python-foo
| XB-Python-Version: X.Y
| Provides: pythonX.Y-foo
`----
But the package does not depend on pythonX.Y, which I think
it should (since we have stated that only X.Y is supported).

Case B:
There is no lower bound, but there is an upper bound
,----
| Package: python-foo
| XB-Python-Version: 2.3
| Provides: python2.3-foo
`----

Hmm. I suppose this is OK, since if we do not support the
current version, we should not be uploading the package.

Case C:
I have a range in the XS-Python-Version

Case C1: The current version is not supported (say, I support 2.4
and 2.5)
,----
| Package: python-foo
| XB-Python-Version: 2.3
| Provides: python2.3-foo
`----
This is wrong.

Case C2: The current version is supported
,----
| Package: python-foo
| XB-Python-Version: 2.3
| Provides: python2.3-foo
`----
This is OK as well.

So, I posit that dh_python, as currently coded, does not
handle case A and C1 correctly.

manoj
--
Sex is like air. It's only a big deal if you can't get any.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Loïc Minier

2006-07-31, 7:30 am

Hi,

(Could you please stop cross-posting to debian-devel? The discussion
belongs to debian-python@ and the list is low traffic.)

On Sun, Jul 30, 2006, Manoj Srivastava wrote:
> I have only one version in XS-Python-Version (say, 2.4)

I think the patch in #378604 enhance this situation with
"XS-Python-Version: 2.4", but I had trouble too with
"XS-Python-Version: 2.3", see thread on debian-python@, "Generation of
"python" dependencies for public extensions versus python2.3".

Also, the behavior seems completely clean for "XS-Python-Version:
current", have a look at "flumotion" which has private modules which
are compiled in place (in /usr/lib/flumotion) for the current version
of Python on install, and are recompiled on upgrades.

Bye,
--
Loïc Minier <lool@dooz.org>

--
To UNSUBSCRIBE, email to debian-python-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org