|
Home > Archive > Debian Developers > January 2007 > restricted sourceless ARM uploads
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
restricted sourceless ARM uploads
|
|
| Aurelien Jarno 2006-12-20, 1:18 pm |
| Hi,
For those who don't know, I have setup 8 emulated ARM build daemons and
started to upload packages. To know why and for more information, see
[1].
These afternoon I saw on #debian-release:
15:52:11 aj | "# unilateral action to run an emulated buildd -- all arm changes sidelined until fixed."
16:18:59 aj | and now aurel32 will presumably fly off the handle
16:22:37 aba | aj: I don't think that is a good idea either if the portersdo uploads - i.e. blacklisting is probably better than whitelisting
16:23:14 aba | aj: and please let e.g. people upload to non-free and experimental, e.g. tbm or kmuto
16:28:16 aj | aba: not at this time
16:28:28 aba | aj: at which time then?
First I am not very happy to learn that from IRC, I would have prefer to
get a mail. However I am very happy to see a reaction on the ARM build
daemons side, and this time really fast.
Usually, you have to wait that a user detect a problem and warns the
build daemon maintainer [2] [3]. And then wait again.
Note that I would have been happy to not start those build daemons, but
given the way the arm build daemons are administrated, something had to
be done.
Anyway, I have been able to fix a few packages that were not built
successfully. For the others, I have extracted from wanna-build a list
of package that build successfully, but have failed to build for
various reasons. I would like an explanation from the arm build daemon
maintainer why they are still on this state:
* Packages that now build fine:
blam_1.8.3-2 (for 40 days)
evolution-sharp_0.11.1-3 (for 9 days)
gcc-h8300-hms_4.1.1-3 (for 7 days)
hdbc-odbc_1.0.1.1 (for 40 days)
hdbc-missingh_1.0.1.1 (for 40 days)
njb-sharp_0.3.0-1 (for 9 days)
muine_0.8.5-1.1 (for 9 days)
* Packages lost from elara (not uploaded for more than 5 days). Note
that it is obvious using [4] to see there is a problem:
complearn-gui_0.9.7-1
grhino_0.15.2-1
iwatch_0.0.12-3
libdiscid_0.1.0-1
libvorbisidec_1.0.2+svn12153-1
lush_1.2.1-3
nec2c_0.6-1
nmap_4.20-1
units-filter_2.6-1
xmms2_0.2DrHouse-3
* Packages that failed to build because the build daemon netwinder is
XXXXed for weeks wrt to /usr/lib/libtasn1.so.3.0.6
cpufire-applet_1.2-2
gnome-session_2.14.3-4
* Package that is wrongly waiting for kaffe-jthreads for more than a
year:
libgconf-java_2.12.3-1
* Package that failed to build because it build depends on a package
not uploaded fast enough. It has never been requeued.
python-gnome_1.4.5-8
* Also those package (and they are plenty others), could be tagged
not-for-us. This would spare some build daemon time, and would ease to
see which packages have really failed to build or not.
cryopid_0.5.9.1-2
dfsbuild_0.99.2
dphys-kernel-packages_20061027-2
gcc-4.0_4.0.3ds1-8
gnat-4.1_4.1.1-19
hs-plugins_0.9.10-3.4
lcdproc_0.5.1-3
libflorist_2006-1
libopenspc_0.3.99a-2
libpfm3-3.2_3.2.060926-2
libx86_0.99-1
linux-modules-di-mips-2.6_1.00
linux-uvc_0.1.0.svn54-4
mit-scheme_7.7.90+20060906-3
openafs_1.4.2-4
pdns-recursor_3.1.4-1
xmms-openspc_0.0.4-2
xen-3.0_3.0.3-0-2
xen-unstable_3.0-unstable+hg11561-1
Also why the packages ctypes and kbedic are in state building for more
than 17 days, without any action from the build daemon maintainer?
Bye,
Aurelien
[1] http://blog.aurel32.net/?p=33
[2] http://lists.debian.org/debian-arm/...1/msg00052.html
[3] http://lists.debian.org/debian-arm/...2/msg00027.html
[4] http://buildd.debian.org/~jeroen/st...cture.php?a=arm
--
.''`. Aurelien Jarno | GPG: 1024D/F1BCDB73
: :' : Debian developer | Electrical Engineer
`. `' aurel32@debian.org | aurelien@aurel32.net
`- people.debian.org/~aurel32 | www.aurel32.net
| |
| Bill Gatliff 2006-12-20, 1:18 pm |
| Wookey wrote:
>On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>
>
>
>Very impressive piece of work aurelien! We ought to discuss if there
>is any significant reason not to use qemu 'machines' instead of actual
>hardware for slower arches.
>
>
As long as there's the occasional test on actual hardware, I don't have
a problem with it.
For the faster arches, i.e. the ARM9 machines and above, I'm thinking
that we should stick with real hardware so there's no question that the
binaries will run properly.
>How fast is each of your build machines in comparison to existing
>buildds?
>
>The offered Hedges machine is more than twice as fast as existing
>buildds and really ought to be brought into the network - it has been
>offered for about 2 months now. I will mail DSA again to see if anyone
>will tell what needs to be done next, by whom, and when. It is very
>rude to people making such offers to give no response at all for such
>a long time. Fortunately bill seems very tolerant of Debian's foibles,
>
>
What choice do I have? I'm not a DD, so it isn't like I have any
authority to do otherwise. 
>and we can at least use it as a private developer-access machine in
>the meantime.
>
>
Indeed.
b.g.
--
Bill Gatliff
bgat@billgatliff.com
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Wookey 2006-12-20, 1:18 pm |
| On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
> Hi,
>
> For those who don't know, I have setup 8 emulated ARM build daemons and
> started to upload packages. To know why and for more information, see
> [1].
Very impressive piece of work aurelien! We ought to discuss if there
is any significant reason not to use qemu 'machines' instead of actual
hardware for slower arches.
How fast is each of your build machines in comparison to existing
buildds?
The offered Hedges machine is more than twice as fast as existing
buildds and really ought to be brought into the network - it has been
offered for about 2 months now. I will mail DSA again to see if anyone
will tell what needs to be done next, by whom, and when. It is very
rude to people making such offers to give no response at all for such
a long time. Fortunately bill seems very tolerant of Debian's foibles,
and we can at least use it as a private developer-access machine in
the meantime.
> Note that I would have been happy to not start those build daemons, but
> given the way the arm build daemons are administrated, something had to
> be done.
Perhaps you are right. Let us all try to keep this discussion
constructive.
> Anyway, I have been able to fix a few packages that were not built
> successfully. For the others, I have extracted from wanna-build a list
> of package that build successfully, but have failed to build for
> various reasons.
<lots of useful info>
Can you update the armbuildd wiki page with this? (I expect you are going to
anyway).
Thanx you for your efforts on the arm port over the last few months -
they have been extremely helpful.
Can we make aurel32 an arm buildd maintainer? I think he has shown
both competence and dedication this year. I'm sure having more than
one maintainer would be helpful. Aurel - do you want to commit to this
task?
Wookey
--
Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679
work: http://www.aleph1.co.uk/ play: http://wookware.org/
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Aurelien Jarno 2006-12-20, 1:18 pm |
| Wookey a écrit :
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.
I think we should include Paul Brook in the discussion, as he written a
lot of parts of the ARM emulation, and is also active in the Debian ARM
port.
Anyway I will say that if we can have fast machine (and there are fast
ARM machines), it is better than a few not to slow emulated ARM machine.
But that machine should be accepted by the DSA. And here is the problem.
> How fast is each of your build machines in comparison to existing
> buildds?
For what I saw they are a bit faster than the Debian build machines. You
can expect to have a ARM v5 running at around 350 MHz. And with plenty
of RAM (256 MB in my case), so that make a huge difference on some packages.
> The offered Hedges machine is more than twice as fast as existing
> buildds and really ought to be brought into the network - it has been
> offered for about 2 months now. I will mail DSA again to see if anyone
> will tell what needs to be done next, by whom, and when. It is very
> rude to people making such offers to give no response at all for such
> a long time. Fortunately bill seems very tolerant of Debian's foibles,
> and we can at least use it as a private developer-access machine in
> the meantime.
>
>
> Perhaps you are right. Let us all try to keep this discussion
> constructive.
>
>
> <lots of useful info>
>
> Can you update the armbuildd wiki page with this? (I expect you are going to
> anyway).
Yes, I will do that later today.
Bye,
Aurelien
--
.''`. Aurelien Jarno | GPG: 1024D/F1BCDB73
: :' : Debian developer | Electrical Engineer
`. `' aurel32@debian.org | aurelien@aurel32.net
`- people.debian.org/~aurel32 | www.aurel32.net
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Aurelien Jarno 2006-12-20, 1:18 pm |
| Wookey a écrit :
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.
>
BTW, I don't think ARM needs emulated build machines. It's just that
it's the fastest ARM machine I found. My two NSLU2 won't have been enough.
Also it seems the current ARM build machines altogether are fast enough
to build all the packages (except for building big packages like the
kernel). But a faster machine would mean that the Vancouver requirements
are satisfied, and also less machines to handle, so less work for the
ARM build daemon maintainer.
What I wanted to demonstrate, is that the percentage of package built by
an architecture does depends on the speed of the build machines, but
also on how they are administrated.
--
.''`. Aurelien Jarno | GPG: 1024D/F1BCDB73
: :' : Debian developer | Electrical Engineer
`. `' aurel32@debian.org | aurelien@aurel32.net
`- people.debian.org/~aurel32 | www.aurel32.net
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Sam Hocevar 2006-12-20, 1:18 pm |
| On Wed, Dec 20, 2006, Bill Gatliff wrote:
> For the faster arches, i.e. the ARM9 machines and above, I'm thinking
> that we should stick with real hardware so there's no question that the
> binaries will run properly.
Pardon me sir, but can that claim that binaries built on so-called
"real hardware" will unquestionably run (as opposed to, if I understand
correcly, binaries built on an emulated platform) be backed up by any
facts, examples, experimentation results or scientific publication?
Kindest regards,
--
Sam.
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Allombert 2006-12-20, 1:18 pm |
| On Wed, Dec 20, 2006 at 05:05:21PM +0000, Wookey wrote:
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.
A very basic reason is that some packages require 1GB of RAM to build
in finite time and there are no arm and m68k buildd with that amount of
RAM.
An alternative is to use distcc+crosscc to a distccd server with 1GB of
RAM.
Cheers,
--
Bill. <ballombe@debian.org>
Imagine a large red swirl here.
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Aurelien Jarno 2006-12-20, 1:18 pm |
| Aurelien Jarno a écrit :
> * Packages that failed to build because the build daemon netwinder is
> XXXXed for weeks wrt to /usr/lib/libtasn1.so.3.0.6
> cpufire-applet_1.2-2
> gnome-session_2.14.3-4
The situation is actually worse than what I expected. This build daemon
seems to loose files from its chroot. I don't have access to it, but
from the build logs I can say that /sbin/MAKEDEV [1] and
/usr/lib/libtasn1.so.3.0.6 [2] are missing.
Other files may be missing. Given that a lot of packages are using
autoconf to detect available libraries, some packages built by this
build daemon may have some disable functionalities.
Bye,
Aurelien
[1]
http://buildd.debian.org/build.php?...3.5.5a.dfsg.1-4
[2]
http://buildd.debian.org/fetch.cgi?...563605&file=log
--
.''`. Aurelien Jarno | GPG: 1024D/F1BCDB73
: :' : Debian developer | Electrical Engineer
`. `' aurel32@debian.org | aurelien@aurel32.net
`- people.debian.org/~aurel32 | www.aurel32.net
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Gatliff 2006-12-20, 1:18 pm |
| Bill:
>A very basic reason is that some packages require 1GB of RAM to build
>in finite time and there are no arm and m68k buildd with that amount of
>RAM.
>
>
Could you try those packages on hedges? (You can get developer access
from Wookey if you need it). Hedges has 512MB real and 1.5GB swap. And
unlike leisner, the netwinders, or nslu2s, it's expandable if needed.
>An alternative is to use distcc+crosscc to a distccd server with 1GB of RAM.
>
>
I've done this before (rebuilt X that way, just as a test!), but it has
similar concerns to the QEMU approach.
But on that point, I've never had any issues with either
distcc+crossgcc--- which I've tested extensively--- or QEMU. But
forcing the use of real hardware wherever possible means (a) you know
for sure, and (b) you have to keep your real hardware maintained. Those
seem like good things.
b.g.
--
Bill Gatliff
bgat@billgatliff.com
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Bill Allombert 2006-12-20, 7:22 pm |
| On Wed, Dec 20, 2006 at 12:32:25PM -0600, Bill Gatliff wrote:
>
> Could you try those packages on hedges? (You can get developer access
> from Wookey if you need it). Hedges has 512MB real and 1.5GB swap. And
> unlike leisner, the netwinders, or nslu2s, it's expandable if needed.
No use, this will simply thrash the box to no end. Try to build openvrml
for example. It failed on europa after 1000 minutes of inactivity:
<http://buildd.debian.org/fetch.cgi?...222284&file=log>
> But on that point, I've never had any issues with either
> distcc+crossgcc--- which I've tested extensively--- or QEMU. But
Neither I have (I use distcc+crossgcc on top of aranym quite
extensively) so I don't see any reason not to use that if that help
the port to be releasable.
> forcing the use of real hardware wherever possible means (a) you know
> for sure, and (b) you have to keep your real hardware maintained. Those
> seem like good things.
They are sure good thing but not at the price of the port not
being released which is the issue here.
Cheers,
--
Bill. <ballombe@debian.org>
Imagine a large blue swirl here.
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Mirco Bauer 2006-12-20, 7:22 pm |
| Hi Aurelien,
I can give some details about the Mono related packages that failed to
build before but worked for you. I reply inline.
On Wed, 2006-12-20 at 17:39 +0100, Aurelien Jarno wrote:
> * Packages that now build fine:
> blam_1.8.3-2 (for 40 days)
blam failed because of a runtime bug (which is specific to the arm port)
in Mono, which can be indentified by throwing random
System.IO.FileNotFoundException, which was fixed in Mono 1.2.2.1 (see
#394418), so a simple rebuild solves this kind of FTBFS on ARM for Mono
based software. Though there are remaining problems still with Mono and
netwinder, which is why the bug isn't closed yet.
> evolution-sharp_0.11.1-3 (for 9 days)
Same thing here as for blam
> gcc-h8300-hms_4.1.1-3 (for 7 days)
> hdbc-odbc_1.0.1.1 (for 40 days)
> hdbc-missingh_1.0.1.1 (for 40 days)
> njb-sharp_0.3.0-1 (for 9 days)
Same thing as for blam
> muine_0.8.5-1.1 (for 9 days)
This one seems to fail trying to generate some WDSL stuff, that tries to
use the internet, which is not always available on some buildds, but was
on your machine seems like.
Can't say anything about the other packages.
--
Regards,
Mirco 'meebey' Bauer
PGP-Key:
http://keyserver.noreply.org/pks/lo...arch=0xEEF946C8
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GIT d s-:+ a-- C++ UL++++$ P L++$>+++$ E- W+++$ N o? K- w++>! O---- M-
V? PS
PE+ Y- PGP++ t 5+ X++ R tv+ b+ DI? D+ G>++ e h! r->++ y?
------END GEEK CODE BLOCK------
| |
| Aurelien Jarno 2006-12-20, 7:22 pm |
| Mirco Bauer a écrit :
> Hi Aurelien,
>
> I can give some details about the Mono related packages that failed to
> build before but worked for you. I reply inline.
>
> On Wed, 2006-12-20 at 17:39 +0100, Aurelien Jarno wrote:
Oops, there is a mistake (copy & paste abuse), this should have been 9
days, as other mono related packages.
[vbcol=seagreen]
> blam failed because of a runtime bug (which is specific to the arm port)
> in Mono, which can be indentified by throwing random
> System.IO.FileNotFoundException, which was fixed in Mono 1.2.2.1 (see
> #394418), so a simple rebuild solves this kind of FTBFS on ARM for Mono
> based software. Though there are remaining problems still with Mono and
> netwinder, which is why the bug isn't closed yet.
When you say "netwinder", could you reproduce it on others netwinder
machines than the build daemon called that way?
FYI this build daemon seems to have some problems (missing files).
--
.''`. Aurelien Jarno | GPG: 1024D/F1BCDB73
: :' : Debian developer | Electrical Engineer
`. `' aurel32@debian.org | aurelien@aurel32.net
`- people.debian.org/~aurel32 | www.aurel32.net
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Russ Allbery 2006-12-21, 1:29 am |
| Aurelien Jarno <aurel32@debian.org> writes:
> * Also those package (and they are plenty others), could be tagged
> not-for-us. This would spare some build daemon time, and would ease to
> see which packages have really failed to build or not.
[...]
> openafs_1.4.2-4
As the OpenAFS co-maintainer, I can confirm that would be the correct
action to take. OpenAFS tries to fail its build fairly quickly with a
clear error message, but it really should just be tagged not-for-us on arm
(and on mips and mipsel). I tried contacting the Packages-arch-specific
maintainers a few times about this a while back without a response.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Kevin Mark 2006-12-21, 7:32 am |
| On Wed, Dec 20, 2006 at 06:40:06PM +0100, Aurelien Jarno wrote:
> Wookey a écrit :
>
> BTW, I don't think ARM needs emulated build machines. It's just that
> it's the fastest ARM machine I found. My two NSLU2 won't have been enough.
>
> Also it seems the current ARM build machines altogether are fast enough
> to build all the packages (except for building big packages like the
> kernel). But a faster machine would mean that the Vancouver requirements
> are satisfied, and also less machines to handle, so less work for the
> ARM build daemon maintainer.
Hi *,
that brings up an interesting issue wrt the vancouver memorandum: it
didn't address software emulation or virtualization. Can qemu and xen be
shown to be reliable for buildd purposes? Does the aforementioned
document need to be modified for these?
cheers,
Kev
--
| .''`. == Debian GNU/Linux == | my web site: |
| : :' : The Universal | debian.home.pipeline.com |
| `. `' Operating System | go to counter.li.org and |
| `- http://www.debian.org/ | be counted! #238656 |
| my keysever: pgp.mit.edu | my NPO: cfsg.org |
| |
| Wouter Verhelst 2006-12-21, 7:26 pm |
| On Wed, Dec 20, 2006 at 06:50:12PM +0100, Sam Hocevar wrote:
> On Wed, Dec 20, 2006, Bill Gatliff wrote:
>
>
> Pardon me sir, but can that claim that binaries built on so-called
> "real hardware" will unquestionably run (as opposed to, if I understand
> correcly, binaries built on an emulated platform) be backed up by any
> facts, examples, experimentation results or scientific publication?
Binaries built on real hardware are built on a machine that uses the
binaries built on same real hardware. I.e., if it works, at least you're
sure it doesn't work because the compiler and the emulator are
bug-compatible.
--
<Lo-lan-do> Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Sam Hocevar 2006-12-21, 7:26 pm |
| On Thu, Dec 21, 2006, Wouter Verhelst wrote:
>
> Binaries built on real hardware are built on a machine that uses the
> binaries built on same real hardware.
This is not true. We have different ARM machines that use different
CPUs and hardware. Given the versatility of the ARM platform, if the
argument "building the binaries on the same machine that uses the
binaries" was valid whatsoever, it would in fact be an argument in
favour of only using a standardised emulated platform.
> I.e., if it works, at least you're sure it doesn't work because the
> compiler and the emulator are bug-compatible.
But you're sure that it's not because the compiler and the CPU are
bug-compatible?
Regards,
--
Sam.
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| Wookey 2006-12-24, 1:39 am |
| On 2006-12-20 19:58 +0100, Bill Allombert wrote:
> On Wed, Dec 20, 2006 at 12:32:25PM -0600, Bill Gatliff wrote:
>
> No use, this will simply thrash the box to no end. Try to build openvrml
> for example. It failed on europa after 1000 minutes of inactivity:
>
> <http://buildd.debian.org/fetch.cgi?...222284&file=log>
This built OK on hedges in about 6 hours. Is a binNMU needed?
(meant to get back about this earlier but forgot I had it running).
Wookey
--
Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK Tel +44 (0) 1223 811679
work: http://www.aleph1.co.uk/ play: http://wookware.org/
--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
| |
| K. Richard Pixley 2007-01-29, 7:23 pm |
| I know I'm late to the party, but one big win about qemu build servers
is that they can be instantly cloned, replicated, and shared. We can't
do that with real hardware.
--rich
--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
|
|
|
|
|