Debian Developers - native packages

Author

native packages

Bartosz Fenski aka fEnIo

2004-04-12, 3:35 pm

Hello.

I noticed that many new maintainers (me too) often made mistake and
built debian-native packages instead of normal one (those with .diff.gz).
In fact, not only new maintainers are making this mistake. I've seen some
changelogs with the following text:

* Doh! I don't know how it happened, but previous package was built as native

I suppose that normal packages are more than 90% of all.

So why don't we provide some additional header or file in debian subdirectory
to mark native packages?

I mean we've got many useful utilities used to build packages.
So maybe dh_make should add additional template file (or header).
Something like debian/native.ex with the following information:

# If you intend to build native package, please rename this file to
# native (without suffix), otherwise just delete it.

Then linda and lintian could easily check if this file exist, and if
not, then they could check if package contains .diff.gz file.
If not -> show error/warning.

This could be handled with some header in control file instead of
additional file in debian subdirectory. I don't know... maybe some
Native: yes/no.

Well that's my suggestion only, I'm not DD and I'm not even good
programmer, so I probably won't provide any code to acomplish it, but
maybe someone else could be so kindly and code it.

I'll be thankful for comments.
For me it would be very useful, because I always check packages with
linda/lintian, and not always notice that there is no diff.gz file.
I suppose that there is more such people.

regards
fEnIo

--
_ Bartosz Feñski | mailto:fenio@o2.pl | pgp:0x13fefc40 | IRC:fEnIo
_|_|_ 32-050 Skawina - G³owackiego 3/15 - w. ma³opolskie - Polska
(0 0) phone:+48602383548 | Slackware - the weakest link
ooO--(_)--Ooo http://skawina.eu.org | JID:fenio@jabber.org | RLU:172001

Henning Makholm

2004-04-12, 4:34 pm

Scripsit Bartosz Fenski aka fEnIo <fenio@o2.pl>

> So why don't we provide some additional header or file in debian subdirectory
> to mark native packages?

Something like making dpkg-source -b refuse to build a native source
package if the version number contains a dash (unless some force
option is given)?

There are 423 .tar.gz files in the current unstable Sources file that
wouldn't have been built then. Many of them look like they are not
really Debian native, and not really numbered -1 upstream either.

> but maybe someone else could be so kindly and code it.

The first step would be to muster a consensus that it is indeed always
wrong to have no .diff.gz for a package with an upstream author
outside Debian. I'm not quite sure such a consensus exists today.

--
Henning Makholm "Punctuation, is? fun!"

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Shaun Jackman

2004-04-12, 5:34 pm

I won't say anything useful here, but simply add a "Me too!". I would
say the following two statements always hold:

version contains '-' implies not Debian native (i.e. orig/diff)
not (version contains '-') implies Debian native (i.e. tar.gz)

If either of these fails, I think lintian/linda should definitely
complain. Can anyone think of a counterexample?

Cheers,
Shaun

On Mon April 12, 2004 12h32, Henning Makholm wrote:
> Scripsit Bartosz Fenski aka fEnIo <fenio@o2.pl>
>
>
> Something like making dpkg-source -b refuse to build a native
> source package if the version number contains a dash (unless some
> force option is given)?
>
> There are 423 .tar.gz files in the current unstable Sources file
> that wouldn't have been built then. Many of them look like they are
> not really Debian native, and not really numbered -1 upstream
> either.
>
>
> The first step would be to muster a consensus that it is indeed
> always wrong to have no .diff.gz for a package with an upstream
> author outside Debian. I'm not quite sure such a consensus exists
> today.

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Chris Cheney

2004-04-12, 5:34 pm

Jeroen van Wolffelaar

2004-04-12, 5:34 pm

On Mon, Apr 12, 2004 at 08:54:16PM +0200, Bartosz Fenski aka fEnIo wrote:
> Then linda and lintian could easily check if this file exist, and if
> not, then they could check if package contains .diff.gz file.
> If not -> show error/warning.

See #216327, but the bug depends on a proper implementation of source
package lintian overrides, which isn't yet finished.

--Jeroen

--
Jeroen van Wolffelaar
Jeroen@wolffelaar.nl (also for Jabber & MSN; ICQ: 33944357)
http://Jeroen.A-Eskwadraat.nl

Bartosz Fenski aka fEnIo

2004-04-12, 5:34 pm

On Mon, Apr 12, 2004 at 08:32:10PM +0100, Henning Makholm wrote:
>
> The first step would be to muster a consensus that it is indeed always
> wrong to have no .diff.gz for a package with an upstream author
> outside Debian. I'm not quite sure such a consensus exists today.

Well as far as I remember Mentors on debian-mentors always ask people to
make non-native packages. In fact that should be probably default
behaviour. There is no reason to build native packages except
situations where program is really Debian specific.

Yeah, I know that sometimes DD's contributes to external projects, but
from my experience having upstream debian/* subdirectory often makes
more trouble than advantages. Sometimes upstream authors release new
version, and debian/* is outdated. Then I have to create diff.gz even
so.

My conclusion is that if non-native packages should be default ones,
then why don't we provide some warnings if someone made native packages
by mistake?

That won't hurt to do `touch debian/native` for natives, but it hurts
when someone won't notice that he made native by mistake.

regards
fEnIo

--
_ Bartosz Feñski | mailto:fenio@o2.pl | pgp:0x13fefc40 | IRC:fEnIo
_|_|_ 32-050 Skawina - G³owackiego 3/15 - w. ma³opolskie - Polska
(0 0) phone:+48602383548 | Slackware - the weakest link
ooO--(_)--Ooo http://skawina.eu.org | JID:fenio@jabber.org | RLU:172001

Bartosz Fenski aka fEnIo

2004-04-12, 5:34 pm

On Mon, Apr 12, 2004 at 10:52:10PM +0200, Jeroen van Wolffelaar wrote:
>
> See #216327, but the bug depends on a proper implementation of source
> package lintian overrides, which isn't yet finished.

Doh, how could I miss it :/
I've searched through lintian/linda's bugs, but didn't notice it.

Well ok, that's what I wanted to see.
Thanks for pointing it out to me.

regards
fEnIo

--
_ Bartosz Feñski | mailto:fenio@o2.pl | pgp:0x13fefc40 | IRC:fEnIo
_|_|_ 32-050 Skawina - G³owackiego 3/15 - w. ma³opolskie - Polska
(0 0) phone:+48602383548 | Slackware - the weakest link
ooO--(_)--Ooo http://skawina.eu.org | JID:fenio@jabber.org | RLU:172001

Joachim Breitner

2004-04-12, 5:34 pm

Hi,

probably just a corner case, but for packages that happen to have no
useful .tar.gz.orig (either because it is distributed as
..zip.foo.rar.uuenc.gpg or because a lot of stuff has to be removed for
legal reasons), it should be ok for the maintainer to just make one
..tar.gz file, and not to "invent" a diff. As a compromise, a
debian/not_native file to stop lintian/linda from complaining is ok I
guess.

(And maybe a debian/really_native for a debian-native package whose
maintainer - for some wired reason - prefers to have a
..diff.gz/tar.gz.orig source package)

nomeata

Am Mo, den 12.04.2004 schrieb Shaun Jackman um 22:28:[color=darkred]
> I won't say anything useful here, but simply add a "Me too!". I would
> say the following two statements always hold:
>
> version contains '-' implies not Debian native (i.e. orig/diff)
> not (version contains '-') implies Debian native (i.e. tar.gz)
>
> If either of these fails, I think lintian/linda should definitely
> complain. Can anyone think of a counterexample?
>
> Cheers,
> Shaun
>
>
> On Mon April 12, 2004 12h32, Henning Makholm wrote:
--
Joachim "nomeata" Breitner
Debian Developer
nomeata@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
JID: joachimbreitner@amessage.de | http://people.debian.org/~nomeata

Nicolas Boullis

2004-04-12, 6:34 pm

Hi,

On Mon, Apr 12, 2004 at 01:28:35PM -0700, Shaun Jackman wrote:
> I won't say anything useful here, but simply add a "Me too!". I would
> say the following two statements always hold:
>
> version contains '-' implies not Debian native (i.e. orig/diff)
> not (version contains '-') implies Debian native (i.e. tar.gz)
>
> If either of these fails, I think lintian/linda should definitely
> complain. Can anyone think of a counterexample?

Yes. Native package foo version 1.2.3 would be source-NMUed as
1.2.3-0.1, and bin-NMUed as 1.2.3-0.0.1.

Cheers,

Nicolas

PS:
Because it is hard to read.
Why should I not answer above the questions?

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Henning Makholm

2004-04-12, 6:35 pm

Scripsit Joachim Breitner <nomeata@debian.org>

> probably just a corner case, but for packages that happen to have no
> useful .tar.gz.orig (either because it is distributed as
> .zip.foo.rar.uuenc.gpg or because a lot of stuff has to be removed for
> legal reasons), it should be ok for the maintainer to just make one
> .tar.gz file, and not to "invent" a diff.

I respectfully disagree. It is common courtesy to be open about what
is the source file distributed by upstream and what is Debian's
additions and modification. It makes life easier for anyone who cares
about the difference and does not make life siginificantly harder for
anyone.

Who cares about the difference, then? Well,

1. The upstream author cares, if he is curious about how we're
treating his brainchild.

2. The next maintainer (or NMUer) will care, as he tries to make sense
of the packaging.

3. A technically savvy user cares, if he is trying to figure out why
the Debian package behaves differently from a package of the
software on another host - or if he is trying to decide whether to
report a bug (with a patch?) to the Debian BTS or directly
upstream.

4. A paranoid user will care, if he has spent a lot of effort auditing
the upstream source code for his own FooOS build and now wants to
verify that the Debian package is trustworthy.

5. Non-Debian users care that Debian offers a well-organised archive
of upstream sources of free software, which can often be compiled
on other platforms. This is kind of peripheral to the value we add,
but it is immensely useful at times (say, when I needed to compile
VCG on a RedHat machine, the upstream ftp site was dead, and the
Debian archive was the only place where source could be found). But
it only works well if one can get *original* source without added
debianisms that cannot easily be found or rolled back.

On the other hand, I don't see any material advantage in not
distinguishing between the original and Debian additions.

--
Henning Makholm "Nej, hvor er vi altså heldige! Længe
leve vor Buxgører Sansibar Bastelvel!"

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Joachim Breitner

2004-04-12, 7:37 pm

Hi,

of course you are right. You (and others) probably got me wrong. The
..diff.gz system is great, availability of original tar balls also. For
the very most packages, this system is fit and ok. What I say is that,
for whatever reasons (for example upstream sources being in some very
strange format) has to create a .tar.gz file from scratch anyway, and
wants to do so, he should be able to create a non-native package without
a .diff.gz.

As _rene_ pointed out, even for him it would make sense to use diff.gz
if he releases debian packages more often than upstream versions come
in. But still - leave it to the maintainer.

I hope I made my point more clear :-)

nomeata

Am Di, den 13.04.2004 schrieb Henning Makholm um 0:05:
> Scripsit Joachim Breitner <nomeata@debian.org>
>
>
> I respectfully disagree. It is common courtesy to be open about what
> is the source file distributed by upstream and what is Debian's
> additions and modification. It makes life easier for anyone who cares
> about the difference and does not make life siginificantly harder for
> anyone.
>
> Who cares about the difference, then? Well,
>
> 1. The upstream author cares, if he is curious about how we're
> treating his brainchild.
>
> 2. The next maintainer (or NMUer) will care, as he tries to make sense
> of the packaging.
>
> 3. A technically savvy user cares, if he is trying to figure out why
> the Debian package behaves differently from a package of the
> software on another host - or if he is trying to decide whether to
> report a bug (with a patch?) to the Debian BTS or directly
> upstream.
>
> 4. A paranoid user will care, if he has spent a lot of effort auditing
> the upstream source code for his own FooOS build and now wants to
> verify that the Debian package is trustworthy.
>
> 5. Non-Debian users care that Debian offers a well-organised archive
> of upstream sources of free software, which can often be compiled
> on other platforms. This is kind of peripheral to the value we add,
> but it is immensely useful at times (say, when I needed to compile
> VCG on a RedHat machine, the upstream ftp site was dead, and the
> Debian archive was the only place where source could be found). But
> it only works well if one can get *original* source without added
> debianisms that cannot easily be found or rolled back.
>
> On the other hand, I don't see any material advantage in not
> distinguishing between the original and Debian additions.
>
> --
> Henning Makholm "Nej, hvor er vi altså heldige! Længe
> leve vor Buxgører Sansibar Bastelvel!"
--
Joachim "nomeata" Breitner
Debian Developer
nomeata@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
JID: joachimbreitner@amessage.de | http://people.debian.org/~nomeata

Rene Engelhard

2004-04-12, 7:37 pm

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Joachim Breitner wrote:
> As _rene_ pointed out, even for him it would make sense to use diff.gz
> if he releases debian packages more often than upstream versions come
> in. But still - leave it to the maintainer.

weeell, I am not the one alone. Not all upstreams have many releases
following each other in short time and for many Debian revisions
come more often than upstream releases...

The specific example I brought was openoffice.org where he have to remove
some stuff (LZW, some non-free docs, ..) and it would be immensly wasteful
making that native.

No, I don't consider this, but think when someone wants to package a
large game and makes this native.....
(see vegastrike-data on how big game data could be...)

Grüße/Regards,

René
- --
.''`. René Engelhard -- Debian GNU/Linux Developer
: :' : http://www.debian.org | http://people.debian.org/~rene/
`. `' rene@debian.org | GnuPG-Key ID: 248AEB73
`- Fingerprint: 41FA F208 28D4 7CA5 19BB 7AD9 F859 90B0 248A EB73

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAeyRS+FmQsCSK63MRAt/LAJ9BIQ1cVaiQUil6M7wbzZfY9EHrbgCfUHmv
p17yctMTGt0MESYnxizoKbQ=
=kdnY
-----END PGP SIGNATURE-----

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Henning Makholm

2004-04-12, 7:37 pm

Scripsit Joachim Breitner <nomeata@debian.org>

> What I say is that, for whatever reasons (for example upstream
> sources being in some very strange format) has to create a .tar.gz
> file from scratch anyway, and wants to do so, he should be able to
> create a non-native package without a .diff.gz.

And I'm saying that you are still wrong. Even if the maintainer has to
repackage the upstream source as an .orig.tar.gz himself, the
distinction between upstream source and Debian additions is still
valid and useful, and the source for the package should still be
uploaded as an .orig.tar.gz/.diff.gz combination.

> But still - leave it to the maintainer.

No. In my opinion it is (or should be) a *bug* if the maintainer
uploads a source package in <name>_<version>.tar.gz format unless it
is *really* a native package, only useful for Debian and without a
separate upstream author.

--
Henning Makholm "De kan rejse hid og did i verden nok så flot
Og er helt fortrolig med alverdens militær"

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Nathanael Nerode

2004-04-12, 8:43 pm

Joachim Breitner wrote:

> Hi,
>
> probably just a corner case, but for packages that happen to have no
> useful .tar.gz.orig (either because it is distributed as
> .zip.foo.rar.uuenc.gpg or because a lot of stuff has to be removed for
> legal reasons), it should be ok for the maintainer to just make one
> .tar.gz file, and not to "invent" a diff.
Actually, it's awfully convenient quite often to have a separate diff and
orig even if the orig has to be "repacked"; I don't think I've ever seen an
orig which was *so* badly packed that I'd abandon upstream's tree entirely,
rather than just repacking upstream's-unpacked-tree-minus-non-free-files
into a .tar.gz.

> As a compromise, a
> debian/not_native file to stop lintian/linda from complaining is ok I
> guess.

--
Make sure your vote will count.
http://www.verifiedvoting.org/

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Joey Hess

2004-04-12, 9:37 pm

Chris Cheney wrote:
> On Mon, Apr 12, 2004 at 08:32:10PM +0100, Henning Makholm wrote:
>
> A more accurate statement would be it is always wrong to have no
> .diff.gz unless the package is only useful for Debian. Not just with an
> upstream author outside Debian. There are very few packages in Debian
> that should be native.

More accurate, perhaps, but much less likely to be a consensus.

--
see shy jo

Theodore Ts'o

2004-04-12, 9:37 pm

On Mon, Apr 12, 2004 at 11:56:05PM +0100, Henning Makholm wrote:
> And I'm saying that you are still wrong. Even if the maintainer has to
> repackage the upstream source as an .orig.tar.gz himself, the
> distinction between upstream source and Debian additions is still
> valid and useful, and the source for the package should still be
> uploaded as an .orig.tar.gz/.diff.gz combination.

There are times when the maintainer has made changes (to binary files,
for example) which cannot be expressed in a .diff.gz file. Diff
doesn't do binary files.

- Ted

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Rene Engelhard

2004-04-12, 9:39 pm

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Theodore Ts'o wrote:
> On Mon, Apr 12, 2004 at 11:56:05PM +0100, Henning Makholm wrote:
>
> There are times when the maintainer has made changes (to binary files,
> for example) which cannot be expressed in a .diff.gz file. Diff
> doesn't do binary files.

Put them uuencoded into the diff, rm the file in clean and uudecode it
during build (or backup it and move back)....

Grüße/Regards,

René
- --
.''`. René Engelhard -- Debian GNU/Linux Developer
: :' : http://www.debian.org | http://people.debian.org/~rene/
`. `' rene@debian.org | GnuPG-Key ID: 248AEB73
`- Fingerprint: 41FA F208 28D4 7CA5 19BB 7AD9 F859 90B0 248A EB73

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAe0a5+FmQsCSK63MRAmfYAJ0W2V+1GcYy
wBfmu6Ct8XH5YgJNgwCeOQ7l
pdWLmQnjqHmVHUD6jdUK0rM=
=f99O
-----END PGP SIGNATURE-----

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Jay Berkenbilt

2004-04-12, 10:33 pm

On Mon, 12 Apr 2004 15:42:45 -0500, Chris Cheney wrote:

> On Mon, Apr 12, 2004 at 08:32:10PM +0100, Henning Makholm wrote:
>
> A more accurate statement would be it is always wrong to have no
> .diff.gz unless the package is only useful for Debian. Not just with an
> upstream author outside Debian. There are very few packages in Debian
> that should be native.

I think I would have to respectfully disagree with that statement.
What if an upstream author who is a Debian developer includes a fully
working debian/ directory along with, say, an rpm .spec file, in the
main sources so that extracting the sources and running
dpkg-buildpackage is all that is required for this person to build the
package? Would a .diff.gz be required in that case? I would argue
that it would not (since there's nothing to diff) even though the
package may be useful outside of Debian. It would also be possible
for the DD who maintains a package to be in close enough cooperation
with the upstream author to have the upstream author maintain a debian
directory whose control file includes the Debian maintainer as
maintainer. Assuming my goal of becoming a DD is eventually met, I
will be participating in both of these arrangements. (In the latter
case, any Debian-specific files would be merged into the upstream
sources, the upstream author would make a release, and I would
immediately create the Debian package.) Clearly someone who did an
NMU would have to generate a .diff.gz, probably with a -0.1 Debian
version, but any package built by the maintainer would require no
modifications at all -- even for a changelog or control file -- from
the upstream version, even though the package is useful outside of
Debian. Maybe I'm splitting hairs, but it seems to me that it would
appropriate to treat this as a native package. It may even be
inappropriate not to.

It seems to me that the criteria should be that if any changes at all,
even including just adding entries to a changelog file or setting the
Maintainer field in a control file, are required from the pristine
extracted upstream source, then a .diff.gz must be there; otherwise, a
..diff.gz must not be there.

On Mon, 12 Apr 2004 13:28:35 -0700, Shaun Jackman wrote:
> I won't say anything useful here, but simply add a "Me too!". I
> would say the following two statements always hold:
>
> version contains '-' implies not Debian native (i.e. orig/diff)
> not (version contains '-') implies Debian native (i.e. tar.gz)
>
> If either of these fails, I think lintian/linda should definitely
> complain. Can anyone think of a counterexample?

Not exactly a counterexample -- more like an almost could have been
counterexample. :-) The libxml-xerces-perl package (upstream:
Xerces-perl) has a dash in its upstream version number (see
xml.apache.org). The current upstream version is 2.3.0-4. The
current Debian package is 2.3.0-4-0.1. This isn't a counter example
because this is not a native package. In principal, the upstream
author could make a native package using his upstream version number
which would create a counterexample. I guess in this type of package,
if the upstream author were a Debian developer and wanted to package
the software as a native package, said developer could change his
version numbering standard. (This could have been 2.3.0.4, for
example.) For what it's worth, this upstream version is most likely
the way it is because version 2.3.0-4 of the Xerces-perl package
depends upon version 2.3.0 of the Xerces-C package. (In general, a
version of Xerces-perl depends upon a specific version of the
lower-level Xerces-C package.)

--
Jay Berkenbilt <ejb@ql.org>
http://www.ql.org/q/

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Nathanael Nerode

2004-04-12, 10:34 pm

Rene Engelhard wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Theodore Ts'o wrote:
>
> Put them uuencoded into the diff, rm the file in clean and uudecode it
> during build (or backup it and move back)....

This is an argument for a different .diff.gz format (one which supports all
manner of changes), certainly.

--
Make sure your vote will count.
http://www.verifiedvoting.org/

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Jay Berkenbilt

2004-04-12, 10:34 pm

> It seems to me that the criteria should be that if any changes at all,
> even including just adding entries to a changelog file or setting the
> Maintainer field in a control file, are required from the pristine
> extracted upstream source, then a .diff.gz must be there; otherwise, a
> .diff.gz must not be there.

Replying to my own post, the situation that Ted Ts'o mentioned where
diff can't represent the changes demonstrates that my statement does
have some exceptions.

I guess all I'm saying is that I don't see "useful outside of Debian"
as relevant to whether the package is native. I only see "changes
required from upstream version" as relevant.

Although it is true that one cannot encapsulate changes to binary
files in the diff directly, one could handle changes to binary files
in a dpatch-style way: create ASCII representations of xdelta files
that are decoded and applied in debian/rules. I'm sure the situation
of doing stuff like adding icons must not be that uncommon in debian
packages, so people must have plenty of solutions to this problem.
Maybe tricks like this would be worth doing in some cases and not in
others. Even this doesn't work if you have to completely exclude some
portion of the upstream source because it can't be redistributed,
though. But I fear that I'm rambling and drifting off topic, so I'll
stop. :-)

--
Jay Berkenbilt <ejb@ql.org>
http://www.ql.org/q/

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Jeroen van Wolffelaar

2004-04-13, 5:38 am

On Mon, Apr 12, 2004 at 09:30:39PM -0400, Jay Berkenbilt wrote:
> What if an upstream author who is a Debian developer includes a fully
> working debian/ directory along with, say, an rpm .spec file, in the
> main sources so that extracting the sources and running
> dpkg-buildpackage is all that is required for this person to build the
> package? Would a .diff.gz be required in that case? I would argue
> that it would not (since there's nothing to diff)

Are you doing an upstream release just because some Debian requirement
changed? That would be required then.

You can still package a .orig.tar.gz containing everything but the
debian dir, and put the debian dir in the diff. That way, you can make
packaging changes without doing an upstream release. What would
non-Debian users of that package think when you release a new version,
with in the changelog only fixes for Debian packaging, no changes to the
program itself? What would Debian think if you made a new release,
because you need to fix the .rpm spec file, but there was no change
whatsoever relevant to Debian?

Of course, nothing against maintaining the debian dir together with
upstream in your version control system, but still packaging as orig +
diff makes sense.

--Jeroen

--
Jeroen van Wolffelaar
Jeroen@wolffelaar.nl (also for Jabber & MSN; ICQ: 33944357)
http://Jeroen.A-Eskwadraat.nl

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Chris Cheney

2004-04-13, 5:38 am

On Mon, Apr 12, 2004 at 08:57:31PM -0400, Theodore Ts'o wrote:
> On Mon, Apr 12, 2004 at 11:56:05PM +0100, Henning Makholm wrote:
>
> There are times when the maintainer has made changes (to binary files,
> for example) which cannot be expressed in a .diff.gz file. Diff
> doesn't do binary files.

This is easily worked around by using diff -a and uuencoding the result.
I use this method with KDE branch updates for the KDE debs. Note that if
you have changes involving UTF-8 code you may have to use diff -a even
though its just "text" since diff in many cases incorrectly identifies
UTF-8 code as binary data. Some people in the past have claimed that
diff does not have this problem... I still think that debian should
default to using diff -a so the extra step wouldn't be required.

Chris

Frank Küster

2004-04-13, 5:38 am

Joachim Breitner <nomeata@debian.org> wrote:

> of course you are right. You (and others) probably got me wrong. The
> .diff.gz system is great, availability of original tar balls also. For
> the very most packages, this system is fit and ok. What I say is that,
> for whatever reasons (for example upstream sources being in some very
> strange format) has to create a .tar.gz file from scratch anyway, and
> wants to do so, he should be able to create a non-native package without
> a .diff.gz.

I disagree. Just assume that you are a user of such a package and want
to try to build a Debian package of a development snapshot. This is by
far easier if you extract the archive you got in the way described in
README.Debian, remove stuff as described in README.Debian (or keep it if
you want to have non-free things, or do whatever you like) and then
(try to) apply the Debian-specific changes of the old version.=20

This procedure will be much harder if there is no difference between the
repacked/crippled upstream archive and the Debianized archive.

> I hope I made my point more clear :-)

Clear, but still wrong in my opinion. Hennings' other points are also
valid, I think

Regards, Frank
--=20
Frank K=FCster, Biozentrum der Univ. Basel
Abt. Biophysikalische Chemie

Roland Stigge

2004-04-13, 5:42 am

Jay Berkenbilt wrote:
> Not exactly a counterexample -- more like an almost could have been
> counterexample. :-) The libxml-xerces-perl package (upstream:
> Xerces-perl) has a dash in its upstream version number (see
> xml.apache.org). The current upstream version is 2.3.0-4. The
> current Debian package is 2.3.0-4-0.1. This isn't a counter example
> because this is not a native package. In principal, the upstream
> author could make a native package using his upstream version number
> which would create a counterexample.

Policy 5.6.11 says about upstream_version: "[...] If there is no
debian_revision then hyphens are not allowed; [...]". So this
"counterexample" isn't Policy-compliant.

bye,
Roland

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Roland Stigge

2004-04-13, 5:42 am

Henning Makholm wrote:
> There are 423 .tar.gz files in the current unstable Sources file that
> wouldn't have been built then. Many of them look like they are not
> really Debian native, and not really numbered -1 upstream either.

This looks like a good reason for a bug. In the past, I filed some of
this kind (though only when I came across one by accident).

bye,
Roland

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Roland Stigge

2004-04-13, 5:42 am

Nicolas Boullis wrote:
>
> Yes. Native package foo version 1.2.3 would be source-NMUed as
> 1.2.3-0.1, and bin-NMUed as 1.2.3-0.0.1.

This case should be possible for lintian/linda to be handled easily.

bye,
Roland

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Henning Makholm

2004-04-13, 9:36 am

Scripsit Theodore Ts'o <tytso@mit.edu>

> There are times when the maintainer has made changes (to binary files,
> for example) which cannot be expressed in a .diff.gz file. Diff
> doesn't do binary files.

Diff does do uuencoded (or base64 encoded) files of any kind. It
makes the build process a little more cumbersome, but it is *not* a
valid reason to deny our users access to upstream's versions of the
files.

--
Henning Makholm "What a hideous colour khaki is."

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Henning Makholm

2004-04-13, 9:38 am

Scripsit Jay Berkenbilt <ejb@ql.org>

> What if an upstream author who is a Debian developer includes a fully
> working debian/ directory along with, say, an rpm .spec file, in the
> main sources so that extracting the sources and running
[...]

> Clearly someone who did an NMU would have to generate a .diff.gz,
> probably with a -0.1 Debian version,

If the package needs to switch back and forth between "native" and
"non-native", it's a sign that one of the classifications is wrong
anyway.

How about this: A package is only *native* is the Debian archive and
mirror network is THE canonical point of distribution for released
versions. NMUs for native packages should be packaged as .tar.gz (no
diff), and the NMU then becomes THE canonically last released version
of the software.

If the author wants to maintain his own "primary" or "original"
distribution site for released versions, then the package is *not
native*, even though the author may be identical to the Debian
maintainer.

There's nothing wrong in principle with upstream releasing a working
debian/ directory in his sources. It just doesn't mean that the
package becomes native. The source package should still be formatted
as .orig.tar.gz plus .diff.gz. If the Debian maintainer decides that
the contents of upstream's debian/ directory are good and do not need
changing, he's free to upload an empty .diff.gz. Then NMUs can be done
without changing the package's status, and there will be no confusion
if the author tires of the package, and different people inherit the
upstream and Debian maintainerships.

--
Henning Makholm "Monarki, er ikke noget materielt ... Borger!"

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Theodore Ts'o

2004-04-14, 2:17 pm

On Tue, Apr 13, 2004 at 03:47:38AM +0200, Rene Engelhard wrote:
>
> Put them uuencoded into the diff, rm the file in clean and uudecode it
> during build (or backup it and move back)....

And when dpkg-source does that automatically for me when building the
package, and when it also automatically reverses the process when the
source package is unpacked, I'll consider it....

Right now, the first time I find out is when try to build the package,
and it bombs out. And if I have to make a uuencoded blob, then I also
have to modify the debian/rules file to uudecode it, which makes the
debian directory a total mess, just for Debian, and just because some
fanatics (probably the same fanatics who don't do any coding and just
like to wank on and on about evil firmware :-P) are complaining that
they want to see changes upstream. Well, if you want to see the
changes from upstream, you can browse the source repository directly:

http://www.thunk.org:5000/

I should note here that I am both the maintainer and the upstream, so
what typically happens is that very often, when I fix a bug in e2fsck,
I also modify a test filesystem, which is binary blob, to create a
testcase demonstrating that the bug has been fixed. So I am not just
introducing a new binary file, such as an icon; it does happen that I
am modifying an existing binary file. Right now diff.gz just doesn't
work for me in many cases. And no, I am not going to temporarily
create kludge-o-rama's in my debian directory just to deal with with
the fact that dpkg-source is broken with respect to changes in binary
files. I will just simply use a Debian native file format instead,
until I can release a new snapshot release that obviates the need for
changes to binary test cases.

- Ted

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Manoj Srivastava

2004-04-14, 2:19 pm

On Mon, 12 Apr 2004 20:57:31 -0400, Theodore Ts'o <tytso@mit.edu> said:

> On Mon, Apr 12, 2004 at 11:56:05PM +0100, Henning Makholm wrote:
[color=darkred]
> There are times when the maintainer has made changes (to binary
> files, for example) which cannot be expressed in a .diff.gz file.
> Diff doesn't do binary files.

implode:
$(checkdir)
-test -d debian/Config && (cd debian && \
tar zfc debian.tar.gz Config && \
uuencode debian.tar.gz debian.tar.gz > debian.uue \
&& rm -f debian.tar.gz )
explode:
$(checkdir)
-test ! -d debian/Config && (cd debian && uudecode debian.uue \
&& tar zfx debian.tar.gz && rm -f debian.tar.gz )

stamp-conf/dist:
$(checkdir)
-test ! -d debian/Config && $(MAKE) -f debian/rules explode
...

manoj
--
McGowan's Madison Avenue Axiom: If an item is advertised as "under
$50", you can bet it's not $19.95.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/%7Esrivasta/>
1024R/C7261095 print CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Theodore Ts'o

2004-04-14, 2:19 pm

On Tue, Apr 13, 2004 at 06:12:54PM -0500, Manoj Srivastava wrote:
>
> [ Makefile suggestion ]
>

This assumes that the binary blob is in the debian directory. It
isn't. It might be in tests/f_dupinode/image.gz, for example. I've
made the change in tests/f_dupinode, and I've checked it into my
source control repository in tests/f_dupinode/image.gz.

Forcing me to keep copies of those binary files that have changed
inside the debian directory is a complete kludge, particularly since
the debian directory is also under source control, and in the end,
when I do a full release, the changed binary file *wants* to be in
tests/f_dupinode/image.gz, not in some random place in the debian
directory.

- Ted

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Henning Makholm

2004-04-14, 2:20 pm

Scripsit Theodore Ts'o <tytso@mit.edu>

> Forcing me to keep copies of those binary files that have changed
> inside the debian directory is a complete kludge,

Who says they have to be in the debian directory? dpkg-source handles
changes in other parts of the source tree just fine.
>
--
Henning Makholm "Den nyttige hjemmedatamat er og forbliver en myte.
Generelt kan der ikke peges på databehandlingsopgaver af
en sådan størrelsesorden og af en karaktér, som berettiger
forestillingerne om den nye hjemme- og husholdningsteknologi."

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Adam Heath

2004-04-14, 7:35 pm

On Wed, 14 Apr 2004, Theodore Ts'o wrote:

> This assumes that the binary blob is in the debian directory. It
> isn't. It might be in tests/f_dupinode/image.gz, for example. I've
> made the change in tests/f_dupinode, and I've checked it into my
> source control repository in tests/f_dupinode/image.gz.
>
> Forcing me to keep copies of those binary files that have changed
> inside the debian directory is a complete kludge, particularly since
> the debian directory is also under source control, and in the end,
> when I do a full release, the changed binary file *wants* to be in
> tests/f_dupinode/image.gz, not in some random place in the debian
> directory.

delete the binary file during debian/rules clean.

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Gunnar Wolf

2004-04-14, 8:35 pm

Joachim Breitner dijo [Tue, Apr 13, 2004 at 12:35:38AM +0200]:
> Hi,
>
> of course you are right. You (and others) probably got me wrong. The
> .diff.gz system is great, availability of original tar balls also. For
> the very most packages, this system is fit and ok. What I say is that,
> for whatever reasons (for example upstream sources being in some very
> strange format) has to create a .tar.gz file from scratch anyway, and
> wants to do so, he should be able to create a non-native package without
> a .diff.gz.

What I have done in suche cases is to repackage the original upstream
directory just as it is distributed from the author. In one case
(can't remember the exact package - it was a NMU I did at debconf3) I
had to modify it (removed a non-free RFC from the orig.tar.gz). That
was clearly documented in README.Debian, pointing to the original
site. That is a compromise you must sometimes make.

--
Gunnar Wolf - gwolf@gwolf.cx - (+52-55)5630-9700 ext. 1366
PGP key 1024D/8BB527AF 2001-10-23
Fingerprint: 0C79 D2D1 2C4E 9CE4 5973 F800 D80E F35A 8BB5 27AF

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Gunnar Wolf

2004-04-14, 9:33 pm

Adam Heath dijo [Wed, Apr 14, 2004 at 05:44:59PM -0500]:
>
> delete the binary file during debian/rules clean.

You would still be distributing it in your orig.tar.gz, and that still
breaks DFSG. You have to compromise the 'pristine sources' ideal (note
that it is not required by Debian, just reccomended) and repackage the
orig.tar.gz without the blobs.

--
Gunnar Wolf - gwolf@gwolf.cx - (+52-55)5630-9700 ext. 1366
PGP key 1024D/8BB527AF 2001-10-23
Fingerprint: 0C79 D2D1 2C4E 9CE4 5973 F800 D80E F35A 8BB5 27AF

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Adam Majer

2004-04-14, 10:34 pm

Henning Makholm wrote:

>5. Non-Debian users care that Debian offers a well-organised archive
> of upstream sources of free software, which can often be compiled
> on other platforms. This is kind of peripheral to the value we add,
> but it is immensely useful at times (say, when I needed to compile
> VCG on a RedHat machine, the upstream ftp site was dead, and the
> Debian archive was the only place where source could be found). But
> it only works well if one can get *original* source without added
> debianisms that cannot easily be found or rolled back.
>
>On the other hand, I don't see any material advantage in not
>distinguishing between the original and Debian additions.
>
>
>
I think there is one: laziness. Most of the time people don't even check
what they are uploading and then there is the "oopss, didn't see that
huge source upload for -15, -16, -17 version of XYZ".

Anyway, I have found some great advantages of having .diff.gz one of
which is at least some bandwidth conservation. For example, if xemacs
has a version,

21.4.15-1
-2
-3
-4
21.4.16-1

Why would someone that wants to rebuild these packages (for whatever
reason), need to DL 10MB of stuff for every single debian release? Why
not just DL 10MB once, then you just need 4 x 100kB.

What about if I want to see the difference between -2 and -3? Do I need
to actually compare the entire darn tree or just the diff?

What if I want to see the difference between upstream .15 and .16?
Without the .orig.tar.gz, I can't.

Maybe we need a Policy change that addresses this so that all packages
that are *not* Debian specific or made for Debian exclusively, are
required to use the .orig.tar.gz + diff.gz package method. Right now
this is only a guideline in the Policy which I believe is insufficient

- Adam

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Theodore Ts'o

2004-04-14, 10:34 pm

On Wed, Apr 14, 2004 at 07:24:22PM -0600, Gunnar Wolf wrote:
>
> You would still be distributing it in your orig.tar.gz, and that still
> breaks DFSG. You have to compromise the 'pristine sources' ideal (note
> that it is not required by Debian, just reccomended) and repackage the
> orig.tar.gz without the blobs.

Nope, it doesn't break DFSG. The binary image is a **filesystem**.
It is used for a regression test suite for e2fsck. Yes I know that
regression test suites is a concept which is foreign to many open
source programmers, but hopefully it is not completely unfamiliar to
people with a smattering of training in software engineering
techiques.

The binary image of the filesystem is in its preferred form for
modifications, using the debugfs tool. And they are shipped with
e2fsprogs and they are part of the "prestine source".

However, when I fix a bug, sometimes I need to modify one of the test
filesystems in order to introduce a new type of filesystem corruptions
so it can be tested by the regression test suite. The general rule of
thumb is that is that I will try to reproduce the problem, create a
simple test case, or modify an existing test case to encompass the
problem, verify that it breaks under the current version of e2fsck,
and then fix the bug in e2fsck, and then verify that the new version
of e2fsck can pass the regression test suite.

The test suite is part of the e2fsprogs sources, and so when I upload
the patch, the fix may include a modified binary image of the test
filesystem --- which dpkg-source cannot deal with. Nevertheless, that
filesystem image is in fact a **source** file from the legalistic
definition of the DFSG --- the filesystem image is in fact the
perferred --- nay, the only --- form of the file for modifications and
updates.

- Ted

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Gunnar Wolf

2004-04-15, 2:33 am

Theodore Ts'o dijo [Wed, Apr 14, 2004 at 09:55:47PM -0400]:
>
> Nope, it doesn't break DFSG. The binary image is a **filesystem**.
> It is used for a regression test suite for e2fsck. Yes I know that
> regression test suites is a concept which is foreign to many open
> source programmers, but hopefully it is not completely unfamiliar to
> people with a smattering of training in software engineering
> techiques.
> (...)

Ugh, I'm sorry - I didn't quite leave the 'binary blobs' thread before
entering this one (:

By the way, this seems like a very nice and interesting battery of
tests. They are present, then, in your .orig.tar.gz? I think I will
take a look into it!

Greetings,

--
Gunnar Wolf - gwolf@gwolf.cx - (+52-55)5630-9700 ext. 1366
PGP key 1024D/8BB527AF 2001-10-23
Fingerprint: 0C79 D2D1 2C4E 9CE4 5973 F800 D80E F35A 8BB5 27AF

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Bartosz Fenski aka fEnIo

2004-04-15, 9:34 am

[...]

Are we going to make some consensus about that?

There are many opinions that orig+diff is the proper solution, so maybe
changing policy is a good idea?

I mean there are almost 40 messages in this thread and I don't see any
decisions...

I think this issues should be handled by Policy. It is very important.

regards
fEnIo

--
_ Bartosz Feñski | mailto:fenio@o2.pl | pgp:0x13fefc40 | IRC:fEnIo
_|_|_ 32-050 Skawina - G³owackiego 3/15 - w. ma³opolskie - Polska
(0 0) phone:+48602383548 | Slackware - the weakest link
ooO--(_)--Ooo http://skawina.eu.org | JID:fenio@jabber.org | RLU:172001

Nathanael Nerode

2004-04-15, 4:34 pm

Gunnar Wolf wrote:

> Adam Heath dijo [Wed, Apr 14, 2004 at 05:44:59PM -0500]:
>
> You would still be distributing it in your orig.tar.gz, and that still
> breaks DFSG. You have to compromise the 'pristine sources' ideal (note
> that it is not required by Debian, just reccomended) and repackage the
> orig.tar.gz without the blobs.

Um, we're talking here about cases where the blobs are the source and really
are the preferred form for modification; these are binary testcases for
manipulation routines, and are presumably edited in a hex editor or some
such.

There are other cases where binaries are definitely the source, such as
pictures.

--
There are none so blind as those who will not see.

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Goswin von Brederlow

2004-04-20, 4:34 am

Theodore Ts'o <tytso@mit.edu> writes:

> On Tue, Apr 13, 2004 at 03:47:38AM +0200, Rene Engelhard wrote:
>
> And when dpkg-source does that automatically for me when building the
> package, and when it also automatically reverses the process when the
> source package is unpacked, I'll consider it....
>
> Right now, the first time I find out is when try to build the package,
> and it bombs out. And if I have to make a uuencoded blob, then I also
> have to modify the debian/rules file to uudecode it, which makes the
> debian directory a total mess, just for Debian, and just because some
> fanatics (probably the same fanatics who don't do any coding and just
> like to wank on and on about evil firmware :-P) are complaining that
> they want to see changes upstream. Well, if you want to see the
> changes from upstream, you can browse the source repository directly:
>
> http://www.thunk.org:5000/
>
> I should note here that I am both the maintainer and the upstream, so

Then just make a new upstream release.

> what typically happens is that very often, when I fix a bug in e2fsck,
> I also modify a test filesystem, which is binary blob, to create a
> testcase demonstrating that the bug has been fixed. So I am not just
> introducing a new binary file, such as an icon; it does happen that I
> am modifying an existing binary file. Right now diff.gz just doesn't
> work for me in many cases. And no, I am not going to temporarily
> create kludge-o-rama's in my debian directory just to deal with with
> the fact that dpkg-source is broken with respect to changes in binary
> files. I will just simply use a Debian native file format instead,
> until I can release a new snapshot release that obviates the need for
> changes to binary test cases.
>
> - Ted

MfG
Goswin

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Theodore Ts'o

2004-04-20, 1:34 pm

On Tue, Apr 20, 2004 at 10:24:53AM +0200, Goswin von Brederlow wrote:
>
> Then just make a new upstream release.
>

Upstream releases need to be more stable and take longer to settle out
than I am willing to wait to fix a bug in unstable. Sorry, not a
solution.

- Ted

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org