|
Home > Archive > Debian Developers > February 2004 > Proposal for an advanced Debian-QA system
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Proposal for an advanced Debian-QA system
|
|
| Bluefuture 2004-02-20, 7:34 am |
| This are some idea for an advanced automatic Debian-qa system:
The Debian-qa and debian devel could develop a system for trace all
upstream version of a software using the Debian/Watch file and uscan
tool and put this info in a db storing relase version, devel/stable
status of the new upstream relase, upstream relase date and (other
useful info?).
Somthing like this:
http://www.distrowatch.com/table.ph...ribution=debian
The package info records on DB could be modified (form a web interface?)
by the package's Mantainer or modifing the watch file in the next upload
version of the package.
So basically Debian Qa, debian Devlopers, and Debian Users could see
what packages are outdated without waiting for a bug labeled "new
upstream release". A very Big view about Debian updated status.
There could be defined alert leves derived from an algorithm based on
factor like:
- missed upstream versions
- bugs collected by package
- date diffs from upstream release and package upload
(stable/testing/sid/experimental/incoming)
- Priorty field in the package
- numer of packages that depend from this one
- end every other useful derivable infos.
So this alert sensors could perform some automatical tasks. For example:
- Defcon 5 Send an automatic alert via email to Debian Qa
- Defcon 4 Send an automatic email to Debian Qa and open an automatic
wishlist or minor bug on the package.
- Defcon 3 Send an automatic alert via email to the Mantainer requesting
info about lack.
- Defcon 2 Debian Qa contact personally the Mantainers for possible
solution to the lack.
- Defcon 1 Global thermonuclear war ;)
This is only a very very rough draft with I hope to open a discussion
about it on this mailing list.
Sorry for my poor english 
Bye,
Blue
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Pedro M. 2004-02-20, 8:34 am |
| Bluefuture escribió:
>This are some idea for an advanced automatic Debian-qa system:
>
>The Debian-qa and debian devel could develop a system for trace all
>upstream version of a software using the Debian/Watch file and uscan
>tool and put this info in a db storing relase version, devel/stable
>status of the new upstream relase, upstream relase date and (other
>useful info?).
>
>Somthing like this:
>http://www.distrowatch.com/table.ph...ribution=debian
>
>The package info records on DB could be modified (form a web interface?)
>by the package's Mantainer or modifing the watch file in the next upload
>version of the package.
>
>So basically Debian Qa, debian Devlopers, and Debian Users could see
>what packages are outdated without waiting for a bug labeled "new
>upstream release". A very Big view about Debian updated status.
>
>There could be defined alert leves derived from an algorithm based on
>factor like:
>
>- missed upstream versions
>- bugs collected by package
>- date diffs from upstream release and package upload
>(stable/testing/sid/experimental/incoming)
>- Priorty field in the package
>- numer of packages that depend from this one
>- end every other useful derivable infos.
>
>So this alert sensors could perform some automatical tasks. For example:
>
>- Defcon 5 Send an automatic alert via email to Debian Qa
>- Defcon 4 Send an automatic email to Debian Qa and open an automatic
>wishlist or minor bug on the package.
>- Defcon 3 Send an automatic alert via email to the Mantainer requesting
>info about lack.
>- Defcon 2 Debian Qa contact personally the Mantainers for possible
>solution to the lack.
>- Defcon 1 Global thermonuclear war ;)
>
>This is only a very very rough draft with I hope to open a discussion
>about it on this mailing list.
>
>Sorry for my poor english
>
>Bye,
>Blue
>
>
>
>
Good ideas. Go ahead with them. Regards.
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Goswin von Brederlow 2004-02-20, 8:34 am |
| Bluefuture <bluefuture@email.it> writes:
> This are some idea for an advanced automatic Debian-qa system:
>
> The Debian-qa and debian devel could develop a system for trace all
> upstream version of a software using the Debian/Watch file and uscan
> tool and put this info in a db storing relase version, devel/stable
> status of the new upstream relase, upstream relase date and (other
> useful info?).
>
> Somthing like this:
> http://www.distrowatch.com/table.ph...ribution=debian
>
> The package info records on DB could be modified (form a web interface?)
> by the package's Mantainer or modifing the watch file in the next upload
> version of the package.
>
> So basically Debian Qa, debian Devlopers, and Debian Users could see
> what packages are outdated without waiting for a bug labeled "new
> upstream release". A very Big view about Debian updated status.
>
> There could be defined alert leves derived from an algorithm based on
> factor like:
>
> - missed upstream versions
> - bugs collected by package
> - date diffs from upstream release and package upload
> (stable/testing/sid/experimental/incoming)
> - Priorty field in the package
> - numer of packages that depend from this one
> - end every other useful derivable infos.
>
> So this alert sensors could perform some automatical tasks. For example:
>
> - Defcon 5 Send an automatic alert via email to Debian Qa
> - Defcon 4 Send an automatic email to Debian Qa and open an automatic
> wishlist or minor bug on the package.
> - Defcon 3 Send an automatic alert via email to the Mantainer requesting
> info about lack.
> - Defcon 2 Debian Qa contact personally the Mantainers for possible
> solution to the lack.
> - Defcon 1 Global thermonuclear war ;)
I think the first action should be a reminder to the maintainer. If
one has to check the watch page one could just as well check the
upstream homepage.
I would also welcome reminders about bugs for my packages. After a
reasonable while and in increasing timespanes without activity on a
bugreport a small reminder could be send.
The feature could be subscribeable, i.e. you have to activate it once
for each package. Any person could subscribe to the feature with his
email address like a mailinglist. Or it could be added to the package
tracking system.
> This is only a very very rough draft with I hope to open a discussion
> about it on this mailing list.
>
> Sorry for my poor english
>
> Bye,
> Blue
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Joerg Wendland 2004-02-20, 9:34 am |
| Bluefuture, on 2004-02-20, 22:37, you wrote:
> The Debian-qa and debian devel could develop a system for trace all
> upstream version of a software using the Debian/Watch file and uscan
> tool and put this info in a db storing relase version, devel/stable
> status of the new upstream relase, upstream relase date and (other
> useful info?).
Having mandatory debian/watch files in each an every package would at
least be a opportunity to detect MIA maintainers. But only if two
premises are met: 1) upstream has to be debian/watch-able and 2)
upstream has to release a new version now and then. These points are
the problems that stand in the way of a regulation that would demand
debian/watch files.
Joerg
--
Joerg "joergland" Wendland
GPG: 51CF8417 FP: 79C0 7671 AFC7 315E 657A F318 57A3 7FBD 51CF 8417
| |
| Bluefuture 2004-02-20, 9:34 am |
| Il ven, 2004-02-20 alle 23:07, Goswin von Brederlow ha scritto:
> Bluefuture <bluefuture@email.it> writes:
>
>
> I think the first action should be a reminder to the maintainer. If
> one has to check the watch page one could just as well check the
> upstream homepage.
>
The system that should run on debian server and that use Debian/Watch
file and uscan tool must upgrade records in the DB automaticaly on a
(daily?) basis.
The Web interface for the Maintainers should be useful only for an
urgent modification to the watch Field in the QA DBMS -
for example:
the upstream author ftp/http server/link is change from the one filled
by the Mantainer in the Watch file of my last uploaded package but i
don't need to upload a new deb package in Deb repos because there isn't
any particular bug to fix and/or any upstream version it is planned to
be relesed in the next times from the upstream author.
Friendly,
Blue
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Kenneth Pronovici 2004-02-20, 9:34 am |
| On Fri, Feb 20, 2004 at 10:37:22PM +0100, Bluefuture wrote:
[...]
> So this alert sensors could perform some automatical tasks. For example:
>
> - Defcon 5 Send an automatic alert via email to Debian Qa
> - Defcon 4 Send an automatic email to Debian Qa and open an automatic
> wishlist or minor bug on the package.
> - Defcon 3 Send an automatic alert via email to the Mantainer requesting
> info about lack.
> - Defcon 2 Debian Qa contact personally the Mantainers for possible
> solution to the lack.
> - Defcon 1 Global thermonuclear war ;)
I'm not sure you can say it's always a bug for Debian's package to be
out-of-sync with the latest upstream release.
For instance, there is a 2.x release of CPAN's HTML::FromText, which is
a complete rewrite by a new upstream maintainer. However, the new
maintainer freely admits that there's no reason to replace the stable
1.x version (currently in Debian) with his less-stable 2.x version
unless we need one of his new features (which we don't, AFAIK).
Getting a bug automatically filed against my package under these
circumstances would annoy me, as I'm already in control of the
situation. (Of course, it's different if someone actually needs the new
features in the 2.x release.)
KEN
--
Kenneth J. Pronovici <pronovic@debian.org>
| |
| Ben Burton 2004-02-20, 9:34 am |
|
> I'm not sure you can say it's always a bug for Debian's package to be
> out-of-sync with the latest upstream release.
>
> For instance, there is a 2.x release of CPAN's HTML::FromText, which is
> a complete rewrite by a new upstream maintainer. However, the new
> maintainer freely admits that there's no reason to replace the stable
> 1.x version (currently in Debian) with his less-stable 2.x version
> unless we need one of his new features (which we don't, AFAIK).
Moreover, upstream will often release several alpha/beta versions before
a proper release, and it doesn't follow at all that debian should have
every alpha/beta release on its servers.
Ben.
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bluefuture 2004-02-20, 10:33 am |
| Il sab, 2004-02-21 alle 00:00, Ben Burton ha scritto:
>
If the 2.x relase is relased stable from the upstream author but you
find it less-stable than 1.x you can post and maintain 2.x in
experimental and for Debian package bugs or 1.x version bugs upgrade new
package version it in the actual repos (stable/unstable).
> Moreover, upstream will often release several alpha/beta versions before
> a proper release, and it doesn't follow at all that debian should have
> every alpha/beta release on its servers.
>
> Ben.
>
When the nightly (or weekly) Automated Debian QA System run if uscan
doesn't run fine the Automate Debian QA System open automatically a
minor bug on every package on that it has found a uscan-problem
requiring a fix in the watch field in the QA DBMS from the package's
Mantainer thorught the Web Interface to the QA DBMS.
The QA DBMS could store an extra field about stable/development stage of
the package or follow only stable upstream requiring at the package
Mantainer to mantaining a correct and upgraded regular
expression/wildcards in the watch file in the package and the watch
Field in the QA DBMS.
Blue.
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Martin Albert 2004-02-20, 11:33 am |
| [cc'ing you because of your Reply-To header (??)]
On Friday 20 February 2004 22:37, Bluefuture wrote:
> This are some idea for an advanced automatic Debian-qa system:
>
> The Debian-qa and debian devel could develop a system for trace all
> upstream version of a software using the Debian/Watch file and uscan
"What is new about it is not necessarily good and what is good about it,
is not necessarily new." (roughly translated from german).
I _know_ when my upstream releases. The most important job of the Debian
maintainer is to cluefully decide whether any released source code is
valuable and benefits our users.
Enforcing that approach with NM would save the archive from a lot of
packages with arguable value that become rightfully easily forgotten.
While i don't get angry when i find a reminder of new upstream as a
wishlist bugreport, i do get angry when i'm forced to pack a new
upstream although the benefit to the user is actually doubtfull.
[The reason to choose Debian over other distros in the old days was that
debian pkgs came from maintainers actually using that software,
shipped with sensible configurations and evtly. additional docs.]
While some of your ideas may be interesting once elaborated, apply to
others the 'technical solution to a social problem' clause - together
with 'what Debian wants to stand for', see also the archive for
'usefullness of the Stable distribution'.
My 2c, have a nice day, martin
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bluefuture 2004-02-20, 1:34 pm |
| The problem could be analyzed in this way.
There is the Debian Frontier composed by debian-developer/mantainers.
Outside the Debian Frontier there are the freesoftware developers.
Some debian-developer are freesoftware developers and some
debian-freesoftware-developer are at the same time the maintainers of
his own developed software (with or without comunity supply).
So the freesoftware developer/comunity develop Alfa,Beta,RC and Stable
relase of their software. They choiche in your own opinion when a
software tool is Stable to relase. So a Stable software exist also
outside Debian Frontier.
The question is. Is a new stable software ready or useful and Stable in
debian for debian Comunity?
Debian is a very big comunity. Over 10.000 packages. Somthing could make
the system not functional in a way of a upgrade. The problem already
exist for the applications still months in unstable waiting for
dependencies. No problem. This is monitorated by "Why this application
is not in testing?".
So why we couldn't create a general overview by the betwen upstream,
that for definitions are external from the debian frontier e the Debian
project internal space of experimental/unstable/incoming(delayed
incoming).
I don't want angry Mantainers, so to be forced to "to pack a new
upstream although the benefit to the user is actually doubtfull."
By an actualizzed overview of the external upstream situation in
comparison to the package in the official repos could give a Great
Overview to the QA if somthing is going wrong, in a 10.000+ packages
system, in the state of healt of a not only stable but also updated
distribution.
A single mantainer cannot - alone - view the general armonization of the
system.
Actually we mointor the internal system?
General Monitor what there is outside our own Debian frontier could be
the next step for Debian?
Friendly,
Blu.
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bluefuture 2004-02-21, 7:34 am |
| There is a ground problem for begin to work on my proposal and it is
well explained in this post from Filippo Giunchedi on debian QA ml:
http://lists.debian.org/debian-qa/2...1/msg00042.html
Il sab, 2004-02-21 alle 09:29, Matt Bonner ha scritto:
> I think the basic resistance you are getting--and will get with any
> version of your
> idea--is that it takes work. Somebody has to implement what you are
> saying. It
> is not clear from your posts whether you intend to do that work. If you
> implement
> something like what you describe, AND maintain it for a while, AND
> people find it
> useful, THEN it will probably become part of Debian.
> As you appear to know, Debian is all volunteer. Things get done because
> someone
> volunteers to do them. Who do you want to volunteer to do your idea?
>
> regards,
> Matt
>
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Adrian Bunk 2004-02-21, 9:33 am |
| On Fri, Feb 20, 2004 at 10:37:22PM +0100, Bluefuture wrote:
> This are some idea for an advanced automatic Debian-qa system:
>...
It's a common misunderstanding that automatic tools would be able to
solve every problem.
Automatic tools might help you to identify QA problems, but they are
only a help for resolving QA problems, not a solution themselves.
> So this alert sensors could perform some automatical tasks. For example:
>
> - Defcon 5 Send an automatic alert via email to Debian Qa
What are the benefits of spamming Debian QA with hundreds of mails?
> - Defcon 4 Send an automatic email to Debian Qa and open an automatic
> wishlist or minor bug on the package.
Never ever open a bug without checking whether a similar bug is already
open.
Never ever automatically send bug reports. It's OK to let a script
automatically generate bug reports, but do not send them unless you
checked every single package whether the issue really exists.
> - Defcon 3 Send an automatic alert via email to the Mantainer requesting
> info about lack.
>...
If the maintainer already told you a good reason why he can't resolve
this issue he'll be quite pissed off if he gets an automated mail
requesting more information.
> Bye,
> Blue
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bluefuture 2004-02-21, 11:33 am |
| Il sab, 2004-02-21 alle 23:54, Adrian Bunk ha scritto:
> On Fri, Feb 20, 2004 at 10:37:22PM +0100, Bluefuture wrote:
>
>
> It's a common misunderstanding that automatic tools would be able to
> solve every problem.
>
I don't belive that automatic tools would be able to solve
Intellectual/Social solvible problems.
> Automatic tools might help you to identify QA problems, but they are
> only a help for resolving QA problems, not a solution themselves.
>
>
> What are the benefits of spamming Debian QA with hundreds of mails?
To take only as example
Defcon 5 ---> A package lacks/skips 3 Stable upstream relases and has 10
important unresolved bugs (??how many bugs could be resolved by an
upstream??) and have many bugs in a state of "forwarded to the upstream
author" on BTS and there are 10 other packages depending on it and it
has an HIGH vote in popularitycontest and there isn't posted any
uploaded version of the actual package in the repos by 4 months etc.
etc.
Do you still want to call this: spamming to QA??
A Mantainer can put the Defcon system on "hold state" posting his
motivation in a filed on the QA DBMS "via Web interface?".
But all the problems (Date lack between upstream and package in debian
repos etc. etc.) could be visible in a table for the comunity and the Qa
purposes. The problems of Alerts (open a bug, mail a report to QA etc.)
are marginal for me.
There will could be only a simple HTML like this
http://developer.skolelinux.no/info...g/distdiff.html that give an
overview of the problems of the packages in unstable for go in testing.
My idea could be a sort of upstream status overview for QA Developer and
Debian Comunity (not a system for systematically piss Mantainers): Why
Debian is not upstream updated?
All this in a clear way.
>
>
> Never ever open a bug without checking whether a similar bug is already
> open.
>
This could be a sort of Bug that result opened by QA: "Your package has
reached Defcon 4.
Plese put on hold on the Defcon monitor system for your package filling
a motivation in the "Web Form" in the QA DBMS for Qa purposes. If a
Mantainer is not "negligent" can set on hold (with varius type of "on
hold" status?? -busy with exams, dependency problems etc.-) the Defcon
system with a valid motivation before package reach the Defcon 4 status.
> Never ever automatically send bug reports. It's OK to let a script
> automatically generate bug reports, but do not send them unless you
> checked every single package whether the issue really exists.
>
>
> If the maintainer already told you a good reason why he can't resolve
> this issue he'll be quite pissed off if he gets an automated mail
> requesting more information.
>
I have explain this point before.
Friendly,
Blu
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Adrian Bunk 2004-02-21, 7:33 pm |
| On Sun, Feb 22, 2004 at 02:20:14AM +0100, Bluefuture wrote:
>...
>
> To take only as example
> Defcon 5 ---> A package lacks/skips 3 Stable upstream relases and has 10
> important unresolved bugs (??how many bugs could be resolved by an
> upstream??) and have many bugs in a state of "forwarded to the upstream
Even a package with over 100 open important bugs might be well
maintained.
There's no direct correlation between the number of open important bugs
and activity of upstream or the Debian maintainer.
> author" on BTS and there are 10 other packages depending on it and it
> has an HIGH vote in popularitycontest and there isn't posted any
> uploaded version of the actual package in the repos by 4 months etc.
> etc.
With these rules, you won't catch many packages with problems.
No matter how you set the rules exactly, you will either get many false
positives or many false negatives.
Besides this, you assume all packages have an active upstream. Debian
contains several packages where the latest upstream release is e.g.
seven or ten years old. Your "advanced Debian-QA system" would
completely ignore such packages.
> Do you still want to call this: spamming to QA??
The main question isn't how to present the information (e.g. mails to
Debian QA or a web page).
The main question is:
How many people are actively working to resolve such issues?
Sending emails alone doesn't has any positive effect.
The opposite is often true:
The more mails go to a list, the more people unsubscribe due to the
traffic, and more important mails get unnoticed in the stream of mails.
> A Mantainer can put the Defcon system on "hold state" posting his
> motivation in a filed on the QA DBMS "via Web interface?".
> But all the problems (Date lack between upstream and package in debian
> repos etc. etc.) could be visible in a table for the comunity and the Qa
> purposes. The problems of Alerts (open a bug, mail a report to QA etc.)
> are marginal for me.
There is additional work required on two sides:
- the maintainers have to inform your system
- people have to go through the reports your system produces
> There will could be only a simple HTML like this
> http://developer.skolelinux.no/info...g/distdiff.html that give an
> overview of the problems of the packages in unstable for go in testing.
No problem with this, but in practice, I don't think it will be used
more than lintian.debian.org .
> My idea could be a sort of upstream status overview for QA Developer and
> Debian Comunity (not a system for systematically piss Mantainers): Why
> Debian is not upstream updated?
>
> All this in a clear way.
There's no problem if you want to set up a web page, but if you want to
do QA work with the results, you need _much_ manual work at processing
the results of the web page.
>
> This could be a sort of Bug that result opened by QA: "Your package has
> reached Defcon 4.
>...
Automatic emails or bug reports don't gain you much - they tend to be
ignored by many maintainers.
> Friendly,
> Blu
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Filippo Giunchedi 2004-02-21, 9:33 pm |
| On Fri, Feb 20, 2004 at 10:37:22PM +0100, Bluefuture wrote:
> This are some idea for an advanced automatic Debian-qa system:
>
> The Debian-qa and debian devel could develop a system for trace all
> upstream version of a software using the Debian/Watch file and uscan
> tool and put this info in a db storing relase version, devel/stable
> status of the new upstream relase, upstream relase date and (other
> useful info?).
please note this check is already implemented in the PTS
(http://packages.qa.debian.org) although not globally but on a per-package
basis. Anyhow it would be easy to extract such informations (PTS stores data
about packages in XML) and sum them in a page or whatever. This issue (more or
less mandatory debian/watch files) has been already discussed on debian-qa times
ago (check the archives) and I'm starting collecting URLs about every package
without watch file to suggest one to package's maintainer (a long, though not
hard, work... if someone is willing to help is welcome )
[snip]
> There could be defined alert leves derived from an algorithm based on
> factor like:
>
> - missed upstream versions
> - bugs collected by package
> - date diffs from upstream release and package upload
> (stable/testing/sid/experimental/incoming)
> - Priorty field in the package
> - numer of packages that depend from this one
> - end every other useful derivable infos.
this sounds interesting!
kind regards,
filippo
--
Filippo Giunchedi
GNU/PG key: 6B79D401
Random signature below:
I never forget a face, but in your case I'll be glad to make an exception.
-- Groucho Marx
| |
| Pedro M. 2004-02-27, 11:34 am |
| Adrian Bunk escribió:
>On Fri, Feb 20, 2004 at 10:37:22PM +0100, Bluefuture wrote:
>
>
>
>
>It's a common misunderstanding that automatic tools would be able to
>solve every problem.
>
>
Automatic tools are inhumen. And the worse way for newbies.
>Automatic tools might help you to identify QA problems, but they are
>only a help for resolving QA problems, not a solution themselves.
>
>
>
I agree. We need human helpers ( and Debian-ers). Regards.
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
|
|
|
|
|