|
Home > Archive > Debian Developers > January 2006 > Autobuilding and the build-arch target, again
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Autobuilding and the build-arch target, again
|
|
| Simon Richter 2006-01-23, 6:14 pm |
| Hi,
while I must admit that I was not at the latest IRC meeting where this
topic came up, I am now bitten by this problem.
I maintain a bunch of kernel modules that can be either patched onto a
kernel tree or built out-of-tree. No problem, have arch:all packages for
patch and source and arch:any for the modules themselves. However, to
build the kernel patch I need full kernel source, while kernel-headers
is sufficient for the module builds.
Now what I'd like to do is to move kernel-source into
Build-Depends-Indep, so the autobuilders don't need to install it.
Wrong. Because the autobuilders invoke (via dpkg-buildpackage) the
"build" target, which is supposed to build everything including the
patch, I now need to have kernel sources around in order to generate a
patch that is then thrown away.
There have been various proposals on that matter, and it always boils
down to the same chicken-and-egg problem:
- policy documents existing practice, which is to invoke "build".
- the existing practice cannot be changed because it would break packages
- the policy cannot be changed because there needs to be an
implementation first.
To summarize the proposals so far:
- "Scan debian/rules, invoke build-arch if present".
Has been tried, does not work.
- "If a package has both Build-Depends and Build-Depends-Indep, it
MUST have a build-arch target"
Would probably catch 95% of all cases. So far, I know no existing
packages that don't have those targets but use both B-D and B-D-I. The
drawback is that there are pretty ugly corner cases where you would have
to jump through hoops to allow build-arch to be called (for example, if
all the binaries you build can be built with just build-essential, but
you need TeX for the documentation)
- "If a package has Build-Depends-Indep and is to be autobuilt, it
MUST have a build-arch target"
This would be special treatment for the autobuilders. Would work in more
cases than the check for both B-D and B-D-I though. As we cannot know
before the build whether a package will build any arch-dependent
packages, we can only guess here. To gain something here, we would have
to check whether any binaries for other architectures already exist, and
only in this case build-arch could be invoked. Talk about ugly.
- "Turn the SHOULD into a MUST in policy and have dpkg-buildpackage
check the Standards-Version"
So far my favourite. It does not break any existing packages and can be
easily implemented. The only drawback is that if your package doesn't
need it and so you ignore the change, someone will NMU your package in
order to bring it up to date with current standards. However the policy
process gets in the way here: For this to work, we need a cutoff
Standards-Version, and to have that, we need to be sure in which version
of policy this will happen, however it is not going to become policy
until we know it works (which is a good meta-policy IMO).
So my proposal to solve this would be:
- create a policy revision that allows building a package with the
"build-arch" and "build-indep" targets. However, the packages still need
to build correctly when the "build" target is invoked, that is, their
build-dependencies need to be complete for this target as well. This
guarantees that we can pull out if anything goes wrong, as any packages
that follow this standard will still build if we revert the change
- if no problems occur, allow build-dependencies that are strictly for
build-indep / binary-indep only to be moved to Build-Depends-Indep. This
will break building those packages with older dpkg-buildpackage unless
care is taken to also install B-D-I, so I'd have no problem at all to
postpone this step until after the etch release.
Questions? Comments? Seconds?
Simon
| |
| Russ Allbery 2006-01-23, 6:14 pm |
| Simon Richter <sjr@debian.org> writes:
> - "If a package has both Build-Depends and Build-Depends-Indep, it
> MUST have a build-arch target"
> Would probably catch 95% of all cases. So far, I know no existing
> packages that don't have those targets but use both B-D and B-D-I.
I know tons of such packages, namely all arch-independent PERL modules
that use debhelper with proper dependencies. You have to put anything
required during debian/rules clean into B-D, not B-D-I.
However, the buildds won't see such packages, so that lack of compliance
with this proposal may not hurt. But then it reduces to your next
proposal.
> - "If a package has Build-Depends-Indep and is to be autobuilt, it
> MUST have a build-arch target"
> This would be special treatment for the autobuilders. Would work in more
> cases than the check for both B-D and B-D-I though. As we cannot know
> before the build whether a package will build any arch-dependent
> packages, we can only guess here. To gain something here, we would have
> to check whether any binaries for other architectures already exist, and
> only in this case build-arch could be invoked. Talk about ugly.
I think the way of phrasing this is "if the source package has both
arch-specific and arch: all binary packages and B-D-I, it must have
build-arch and build-indep targets." It's a pain to have the policy be
this conditional, but I think this is going to make the fewest packages
buggy while still accomplishing the goal.
> - "Turn the SHOULD into a MUST in policy and have dpkg-buildpackage
> check the Standards-Version"
Checking the standards version for this sort of thing seems rather evil to
me, but maybe I'm too conservative.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Michael Banck 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 06:36:40PM +0100, Simon Richter wrote:
> To summarize the proposals so far:
>
> - "Scan debian/rules, invoke build-arch if present".
>
> Has been tried, does not work.
AFAIK it is working as long as you assume debian/rules to be a Makefile,
which is a pretty safe assumption considered it is mandated by policy.
We had a talk about this in #debian-tech some days ago, and the general
consensus seemed to be that somebody should submit a patch for that to
dpkg.
I have attached the log.
cheers,
Michael
| |
| Wouter Verhelst 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 06:59:55PM +0100, Michael Banck wrote:
> On Mon, Jan 23, 2006 at 06:36:40PM +0100, Simon Richter wrote:
>
> AFAIK it is working as long as you assume debian/rules to be a Makefile,
No, that is not true. The code to do it that way had been added to dpkg
1.10.11 (from 2003!), but was pulled in 1.10.15, with the following
changelog:
dpkg (1.10.15) unstable; urgency=low
* Fix detection of va_copy.
* Back out debian/rules build-arch detection. It is *not* possible *at
all* to detect available targets in a rules file. Period.
-- Adam Heath <doogie@debian.org> Fri, 19 Sep 2003 20:02:19 -0500
--
..../ -/ ---/ .--./ / .--/ .-/ .../ -/ ../ -./ --./ / -.--/ ---/ ..-/ .-./ / -/
.../ --/ ./ / .--/ ../ -/ ..../ / -../ ./ -.-./ ---/ -../ ../ -./ --./ / --/
-.--/ / .../ ../ --./ -./ .-/ -/ ..-/ .-./ ./ .-.-.-/ / --/ ---/ .-./ .../ ./ /
.../ .../ / ---/ ..-/ -/ -../ .-/ -/ ./ -../ / -/ ./ -.-./ ..../ -./ ---/ .-../
---/ --./ -.--/ / .-/ -./ -.--/ .--/ .-/ -.--/ .-.-.-/ / ...-.-/
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Frank Lichtenheld 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 06:36:40PM +0100, Simon Richter wrote:
> Questions? Comments? Seconds?
Yet another proposal to solve this problem can be found in #229357
Gruesse,
--
Frank Lichtenheld <djpig@debian.org>
www: http://www.djpig.de/
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Allombert 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 06:45:07PM +0100, Simon Richter wrote:
> There have been various proposals on that matter, and it always boils
> down to the same chicken-and-egg problem:
>
> - policy documents existing practice, which is to invoke "build".
> - the existing practice cannot be changed because it would break packages
> - the policy cannot be changed because there needs to be an
> implementation first.
>
> To summarize the proposals so far:
You forget my proposal in bug #218893 to add a Build-Options control
field indicating if the build-arch target exist, and the patch in
bug #229357.
So far I am waiting a decision from the dpkg maintainers.
At the time the dpkg maintainers were strongly opposed to requiring
the build-arch target but maybe things have changed.
Cheers,
Bill.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Michael Banck 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 07:31:08PM +0100, Wouter Verhelst wrote:
> On Mon, Jan 23, 2006 at 06:59:55PM +0100, Michael Banck wrote:
>
> No, that is not true. The code to do it that way had been added to dpkg
> 1.10.11 (from 2003!), but was pulled in 1.10.15, with the following
> changelog:
Yeah, I know (it is quoted in the URL referenced in the IRC log). In
the same URL however, is the proposal from keybuk to use make -pn
instead of make -qn (the latter got used and later reverted in dpkg).
The difference between the two is that -q checks whether a target is
uptodate and return an appropriate exit code, while -p prints out the
data base (i.e. the rules and variables) that results from reading the
Makefile. The latter seems more robust to me, so we should reevaluate
the "It is *not* possible *at all* to detect available targets in a
rules file." assertion, IMHO.
Michael
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Manoj Srivastava 2006-01-23, 6:14 pm |
| On Mon, 23 Jan 2006 19:31:08 +0100, Wouter Verhelst <wouter@debian.org> said:
> On Mon, Jan 23, 2006 at 06:59:55PM +0100, Michael Banck wrote:
[vbcol=seagreen]
> No, that is not true. The code to do it that way had been added to
> dpkg
> 1.10.11 (from 2003!), but was pulled in 1.10.15, with the following
> changelog:
> dpkg (1.10.15) unstable; urgency=low
> * Fix detection of va_copy.
> * Back out debian/rules build-arch detection. It is *not*
> possible *at all* to detect available targets in a rules file.
> Period.
> -- Adam Heath <doogie@debian.org> Fri, 19 Sep 2003 20:02:19 -0500
Err, don't believe all you read.
========================================
==============================
__> cat notargetfoo.mk
all:
@echo nothing to see here
__> cat withfoo.mk
all: foo
@echo something to see here
foo:
@echo doing foo
__> make -n -p -f withfoo.mk | grep foo: >|/dev/null; echo $?
0
__> make -n -p -f notargetfoo.mk | grep foo: >|/dev/null; echo $?
1
========================================
==============================
Of course, this kinda presupposes that ./debian/rules is a
makefile -- which is what poicy requires it to be. So we can indeed
detect if a target foo exists -- just that dpkg folks were trying not
to depend on the rules file actually being a makefile, as opposed to
something merely having a similar api to a real make file.
manoj
--
The sheep that fly over your head are soon to land.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/%7Esrivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Allombert 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 07:45:10PM +0100, Michael Banck wrote:
> On Mon, Jan 23, 2006 at 07:31:08PM +0100, Wouter Verhelst wrote:
>
> Yeah, I know (it is quoted in the URL referenced in the IRC log). In
> the same URL however, is the proposal from keybuk to use make -pn
> instead of make -qn (the latter got used and later reverted in dpkg).
>
> The difference between the two is that -q checks whether a target is
> uptodate and return an appropriate exit code, while -p prints out the
> data base (i.e. the rules and variables) that results from reading the
> Makefile. The latter seems more robust to me, so we should reevaluate
> the "It is *not* possible *at all* to detect available targets in a
> rules file." assertion, IMHO.
I would strongly suggest you to consider the "Build-Options: build-arch"
solution I proposed that is going to be much more robust than any
parsing of debian/rules. I am afraid the make -pn attempt will fail
and that will postpone the resolution of this issue even more, like
the -qn did.
Beside that, the "Build-Options: build-arch" has been the product of a
long discussion on debian-policy, does not assume debian/rules is a
makefile and provide a general way to expand the dpkg building interface.
Cheers,
--
Bill. <ballombe@debian.org>
Imagine a large red swirl here.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Paul TBBle Hampson 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 07:31:08PM +0100, Wouter Verhelst wrote:
> On Mon, Jan 23, 2006 at 06:59:55PM +0100, Michael Banck wrote:
[vbcol=seagreen]
> No, that is not true. The code to do it that way had been added to dpkg
> 1.10.11 (from 2003!), but was pulled in 1.10.15, with the following
> changelog:
> dpkg (1.10.15) unstable; urgency=low
> * Fix detection of va_copy.
> * Back out debian/rules build-arch detection. It is *not* possible *at
> all* to detect available targets in a rules file. Period.
> -- Adam Heath <doogie@debian.org> Fri, 19 Sep 2003 20:02:19 -0500
Given that a false-negative would just fallback to build (I hope???
That's surely the only sane option) then this issue must have been due
to false positives...
Can anyone contruct a file of some kind that will give a false-positive
for the various proposed detection routines?
Bonus points if it's a Makefile, since that would be a more powerful
argument for it being impossible...
--
-----------------------------------------------------------
Paul "TBBle" Hampson, BSc, MCSE
On-hiatus Asian Studies student, ANU
The Boss, Bubblesworth Pty Ltd (ABN: 51 095 284 361)
Paul.Hampson@Anu.edu.au
"No survivors? Then where do the stories come from I wonder?"
-- Capt. Jack Sparrow, "Pirates of the Caribbean"
License: http://creativecommons.org/licenses/by/2.1/au/
-----------------------------------------------------------
| |
| Wouter Verhelst 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 02:17:36PM -0600, Bill Allombert wrote:
> On Mon, Jan 23, 2006 at 07:45:10PM +0100, Michael Banck wrote:
>
> I would strongly suggest you to consider the "Build-Options: build-arch"
> solution I proposed that is going to be much more robust than any
> parsing of debian/rules.
And I would strongly suggest you to consider Simon Richter's proposal,
which sounds a lot cleaner to me: if you have build-depends-indep in
your debian/control file, you must also implement build-arch and/or
build-indep.
Additionally, that has the advantage that most packages probably already
implement it that way.
--
..../ -/ ---/ .--./ / .--/ .-/ .../ -/ ../ -./ --./ / -.--/ ---/ ..-/ .-./ / -/
.../ --/ ./ / .--/ ../ -/ ..../ / -../ ./ -.-./ ---/ -../ ../ -./ --./ / --/
-.--/ / .../ ../ --./ -./ .-/ -/ ..-/ .-./ ./ .-.-.-/ / --/ ---/ .-./ .../ ./ /
.../ .../ / ---/ ..-/ -/ -../ .-/ -/ ./ -../ / -/ ./ -.-./ ..../ -./ ---/ .-../
---/ --./ -.--/ / .-/ -./ -.--/ .--/ .-/ -.--/ .-.-.-/ / ...-.-/
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Allombert 2006-01-23, 6:14 pm |
| On Mon, Jan 23, 2006 at 11:08:00PM +0100, Wouter Verhelst wrote:
> And I would strongly suggest you to consider Simon Richter's proposal,
> which sounds a lot cleaner to me: if you have build-depends-indep in
> your debian/control file, you must also implement build-arch and/or
> build-indep.
>
> Additionally, that has the advantage that most packages probably already
> implement it that way.
There used to be the issue that the dpkg maintainers were strongly
opposed to that, because that break backward compatibility.
Cheers,
Bill.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Adeodato Simó 2006-01-23, 8:49 pm |
| * Wouter Verhelst [Mon, 23 Jan 2006 23:08:00 +0100]:
> And I would strongly suggest you to consider Simon Richter's proposal,
> which sounds a lot cleaner to me: if you have build-depends-indep in
> your debian/control file, you must also implement build-arch and/or
> build-indep.
> Additionally, that has the advantage that most packages probably already
> implement it that way.
It sounded sensible to me at first, but alas:
<dato> GyrosGeier: oh, but I actually like your proposal as well.
only, how many packages would benefit from having buildds
call 'build-arch' on them, that have _no_ need for B-D-I?
<GyrosGeier> dato, I think that would be very few
[...]
<GyrosGeier> dato, these won't benefit from the implementation directly.
<dato> GyrosGeier: right; but if there's a need for it, a solution for all
would be prefereable, since this would get done probably only once.
<dato> GyrosGeier: (iow, if your proposal is demonstrated to cover 90+% of
the relevant cases, cool, I think one can consider it; but if
it doesn't, it'd be better to aim for a solution that did,
don't you think?)
So with that proposal you're restricting the number of packages that
can benefit from it. Now, if one shows that such restriction does not
have practical impact... (Or that is considerably less bad that the
"mega-impact" that requiring build-* in every package is, that too.)
Cheers,
--
Adeodato Simó dato at net.com.org.es
Debian Developer adeodato at debian.org
An economist is an expert who will know tomorrow why the things he
predicted yesterday didn't happen today.
-- Laurence J. Peter
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Allombert 2006-01-24, 6:24 pm |
| On Tue, Jan 24, 2006 at 07:46:38AM +1100, Paul TBBle Hampson wrote:
>
>
>
> Given that a false-negative would just fallback to build (I hope???
> That's surely the only sane option) then this issue must have been due
> to false positives...
Fallbacking to 'debian/rules build' will cause a FTBFS, because the
buildd will only install Build-Depends and 'debian/rules build' might
also require Build-Depends-Indep.
Cheers,
--
Bill. <ballombe@debian.org>
Imagine a large red swirl here.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Wouter Verhelst 2006-01-29, 9:38 pm |
| On Mon, Jan 23, 2006 at 11:21:55PM +0100, Bill Allombert wrote:
> On Mon, Jan 23, 2006 at 11:08:00PM +0100, Wouter Verhelst wrote:
>
> There used to be the issue that the dpkg maintainers were strongly
> opposed to that, because that break backward compatibility.
Yes, but only if packages who declare build-depends-indep without having
build-arch exist. Anyone feel like finding that out? ;-)
--
..../ -/ ---/ .--./ / .--/ .-/ .../ -/ ../ -./ --./ / -.--/ ---/ ..-/ .-./ / -/
.../ --/ ./ / .--/ ../ -/ ..../ / -../ ./ -.-./ ---/ -../ ../ -./ --./ / --/
-.--/ / .../ ../ --./ -./ .-/ -/ ..-/ .-./ ./ .-.-.-/ / --/ ---/ .-./ .../ ./ /
.../ .../ / ---/ ..-/ -/ -../ .-/ -/ ./ -../ / -/ ./ -.-./ ..../ -./ ---/ .-../
---/ --./ -.--/ / .-/ -./ -.--/ .--/ .-/ -.--/ .-.-.-/ / ...-.-/
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Adeodato Simó 2006-01-29, 9:38 pm |
| * Wouter Verhelst [Wed, 25 Jan 2006 10:14:26 +0100]:
> Yes, but only if packages who declare build-depends-indep without having
> build-arch exist. Anyone feel like finding that out? ;-)
Simon run a test through the archive prior to sending his mail, to
find out.
--
Adeodato Simó dato at net.com.org.es
Debian Developer adeodato at debian.org
A lie can go round the world before the truth has got its boots on.
-- Terry Pratchett
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
|
|
|
|
|