Hi,

For those who don't know, I have setup 8 emulated ARM build daemons and
started to upload packages. To know why and for more information, see
[1].

These afternoon I saw on #debian-release:

15:52:11  aj | "# unilateral action to run an emulated buildd -- all arm cha
nges sidelined until fixed."
16:18:59  aj | and now aurel32 will presumably fly off the handle
16:22:37 aba | aj: I don't think that is a good idea either if the portersdo
 uploads - i.e. blacklisting is probably better than whitelisting
16:23:14 aba | aj: and please let e.g. people upload to non-free and experim
ental, e.g. tbm or kmuto
16:28:16  aj | aba: not at this time
16:28:28 aba | aj: at which time then?

First I am not very happy to learn that from IRC, I would have prefer to
get a mail. However I am very happy to see a reaction on the ARM build
daemons side, and this time really fast.

Usually, you have to wait that a user detect a problem and warns the
build daemon maintainer [2] [3]. And then wait again.

Note that I would have been happy to not start those build daemons, but
given the way the arm build daemons are administrated, something had to 
be done.

Anyway, I have been able to fix a few packages that were not built
successfully. For the others, I have extracted from wanna-build a list
of package that build successfully, but have failed to build for 
various reasons. I would like an explanation from the arm build daemon
maintainer why they are still on this state:

* Packages that now build fine:
blam_1.8.3-2 (for 40 days)
evolution-sharp_0.11.1-3 (for 9 days)
gcc-h8300-hms_4.1.1-3 (for 7 days)
hdbc-odbc_1.0.1.1 (for 40 days) 
hdbc-missingh_1.0.1.1 (for 40 days)
njb-sharp_0.3.0-1 (for 9 days)
muine_0.8.5-1.1 (for 9 days)
 
* Packages lost from elara (not uploaded for more than 5 days). Note
that it is obvious using [4] to see there is a problem:
complearn-gui_0.9.7-1 
grhino_0.15.2-1 
iwatch_0.0.12-3 
libdiscid_0.1.0-1 
libvorbisidec_1.0.2+svn12153-1 
lush_1.2.1-3 
nec2c_0.6-1 
nmap_4.20-1 
units-filter_2.6-1 
xmms2_0.2DrHouse-3 
 
* Packages that failed to build because the build daemon netwinder is
XXXXed for weeks wrt to /usr/lib/libtasn1.so.3.0.6
cpufire-applet_1.2-2 
gnome-session_2.14.3-4 

* Package that is wrongly waiting for kaffe-jthreads for more than a 
year:
libgconf-java_2.12.3-1 
 
* Package that failed to build because it build depends on a package
not uploaded fast enough. It has never been requeued.
python-gnome_1.4.5-8 
 
* Also those package (and they are plenty others), could be tagged
not-for-us. This would spare some build daemon time, and would ease to
see which packages have really failed to build or not.
cryopid_0.5.9.1-2
dfsbuild_0.99.2
dphys-kernel-packages_20061027-2
gcc-4.0_4.0.3ds1-8
gnat-4.1_4.1.1-19
hs-plugins_0.9.10-3.4
lcdproc_0.5.1-3
libflorist_2006-1
libopenspc_0.3.99a-2
libpfm3-3.2_3.2.060926-2
libx86_0.99-1
linux-modules-di-mips-2.6_1.00
linux-uvc_0.1.0.svn54-4
mit-scheme_7.7.90+20060906-3
openafs_1.4.2-4
pdns-recursor_3.1.4-1
xmms-openspc_0.0.4-2
xen-3.0_3.0.3-0-2
xen-unstable_3.0-unstable+hg11561-1

Also why the packages ctypes and kbedic are in state building for more
than 17 days, without any action from the build daemon maintainer?

Bye,
Aurelien

[1] http://blog.aurel32.net/?p=33
[2] http://lists.debian.org/debian-arm/...1/msg00052.html
[3] http://lists.debian.org/debian-arm/...2/msg00027.html
[4] http://buildd.debian.org/~jeroen/st...cture.php?a=arm

-- 
.''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
: :' :  Debian developer           | Electrical Engineer
`. `'   aurel32@debian.org         | aurelien@aurel32.net
`-    people.debian.org/~aurel32 | www.aurel32.net

[ Post a follow-up to this message ]

Wookey wrote:

>On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
>
> 
>
>Very impressive piece of work aurelien! We ought to discuss if there
>is any significant reason not to use qemu 'machines' instead of actual
>hardware for slower arches.
>
>

As long as there's the occasional test on actual hardware, I don't have
a problem with it.

For the faster arches, i.e. the ARM9 machines and above, I'm thinking
that we should stick with real hardware so there's no question that the
binaries will run properly.

>How fast is each of your build machines in comparison to existing
>buildds?
>
>The offered Hedges machine is more than twice as fast as existing
>buildds and really ought to be brought into the network - it has been
>offered for about 2 months now. I will mail DSA again to see if anyone
>will tell what needs to be done next, by whom, and when. It is very
>rude to people making such offers to give no response at all for such
>a long time. Fortunately bill seems very tolerant of Debian's foibles,
>
>

What choice do I have? I'm not a DD, so it isn't like I have any
authority to do otherwise. 


>and we can at least use it as a private developer-access machine in
>the meantime.
>
>

Indeed.



b.g.

--
Bill Gatliff
bgat@billgatliff.com


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

On 2006-12-20 17:39 +0100, Aurelien Jarno wrote:
> Hi,
>
> For those who don't know, I have setup 8 emulated ARM build daemons and
> started to upload packages. To know why and for more information, see
> [1].

Very impressive piece of work aurelien! We ought to discuss if there
is any significant reason not to use qemu 'machines' instead of actual
hardware for slower arches.

How fast is each of your build machines in comparison to existing
buildds?

The offered Hedges machine is more than twice as fast as existing
buildds and really ought to be brought into the network - it has been
offered for about 2 months now. I will mail DSA again to see if anyone
will tell what needs to be done next, by whom, and when. It is very
rude to people making such offers to give no response at all for such
a long time. Fortunately bill seems very tolerant of Debian's foibles,
and we can at least use it as a private developer-access machine in
the meantime.

> Note that I would have been happy to not start those build daemons, but
> given the way the arm build daemons are administrated, something had to
> be done.

Perhaps you are right. Let us all try to keep this discussion
constructive.

> Anyway, I have been able to fix a few packages that were not built
> successfully. For the others, I have extracted from wanna-build a list
> of package that build successfully, but have failed to build for
> various reasons.

<lots of useful info>

Can you update the armbuildd wiki page with this? (I expect you are going to
anyway).

Thanx you for your efforts on the arm port over the last few months -
they have been extremely helpful.

Can we make aurel32 an arm buildd maintainer? I think he has shown
both competence and dedication this year. I'm sure having more than
one maintainer would be helpful. Aurel - do you want to commit to this
task?

Wookey
--
Aleph One Ltd, Bottisham, CAMBRIDGE, CB5 9BA, UK  Tel +44 (0) 1223 811679
work: http://www.aleph1.co.uk/                 play: http://wookware.org/


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

Wookey a écrit :
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote: 
>
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.

I think we should include Paul Brook in the discussion, as he written a
lot of parts of the ARM emulation, and is also active in the Debian ARM
port.

Anyway I will say that if we can have fast machine (and there are fast
ARM machines), it is better than a few not to slow emulated ARM machine.
But that machine should be accepted by the DSA. And here is the problem.

> How fast is each of your build machines in comparison to existing
> buildds?

For what I saw they are a bit faster than the Debian build machines. You
can expect to have a ARM v5 running at around 350 MHz. And with plenty
of RAM (256 MB in my case), so that make a huge difference on some packages.

> The offered Hedges machine is more than twice as fast as existing
> buildds and really ought to be brought into the network - it has been
> offered for about 2 months now. I will mail DSA again to see if anyone
> will tell what needs to be done next, by whom, and when. It is very
> rude to people making such offers to give no response at all for such
> a long time. Fortunately bill seems very tolerant of Debian's foibles,
> and we can at least use it as a private developer-access machine in
> the meantime.
> 
>
> Perhaps you are right. Let us all try to keep this discussion
> constructive.
> 
>
> <lots of useful info>
>
> Can you update the armbuildd wiki page with this? (I expect you are going 
to
> anyway).

Yes, I will do that later today.

Bye,
Aurelien

--
.''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
: :' :  Debian developer           | Electrical Engineer
`. `'   aurel32@debian.org         | aurelien@aurel32.net
`-    people.debian.org/~aurel32 | www.aurel32.net


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

Wookey a écrit :
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote: 
>
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.
>

BTW, I don't think ARM needs emulated build machines. It's just that
it's the fastest ARM machine I found. My two NSLU2 won't have been enough.

Also it seems the current ARM build machines altogether are fast enough
to build all the packages (except for building big packages like the
kernel). But a faster machine would mean that the Vancouver requirements
are satisfied, and also less machines to handle, so less work for the
ARM build daemon maintainer.

What I wanted to demonstrate, is that the percentage of package built by
an architecture does depends on the speed of the build machines, but
also on how they are administrated.

--
.''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
: :' :  Debian developer           | Electrical Engineer
`. `'   aurel32@debian.org         | aurelien@aurel32.net
`-    people.debian.org/~aurel32 | www.aurel32.net


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

On Wed, Dec 20, 2006, Bill Gatliff wrote:

> For the faster arches, i.e. the ARM9 machines and above, I'm thinking
> that we should stick with real hardware so there's no question that the
> binaries will run properly.

Pardon me sir, but can that claim that binaries built on so-called
"real hardware" will unquestionably run (as opposed to, if I understand
correcly, binaries built on an emulated platform) be backed up by any
facts, examples, experimentation results or scientific publication?

Kindest regards,
--
Sam.


--
To UNSUBSCRIBE, email to debian-arm-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

On Wed, Dec 20, 2006 at 05:05:21PM +0000, Wookey wrote:
> On 2006-12-20 17:39 +0100, Aurelien Jarno wrote: 
>
> Very impressive piece of work aurelien! We ought to discuss if there
> is any significant reason not to use qemu 'machines' instead of actual
> hardware for slower arches.

A very basic reason is that some packages require 1GB of RAM to build
in finite time and there are no arm and m68k buildd with that amount of
RAM.

An alternative is to use distcc+crosscc to a distccd server with 1GB of
RAM.

Cheers,
--
Bill. <ballombe@debian.org>

Imagine a large red swirl here.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

Aurelien Jarno a écrit :
> * Packages that failed to build because the build daemon netwinder is
>   XXXXed for weeks wrt to /usr/lib/libtasn1.so.3.0.6
>    cpufire-applet_1.2-2
>    gnome-session_2.14.3-4

The situation is actually worse than what I expected. This build daemon
seems to loose files from its chroot. I don't have access to it, but
from the build logs I can say that /sbin/MAKEDEV [1] and
/usr/lib/libtasn1.so.3.0.6 [2] are missing.

Other files may be missing. Given that a lot of packages are using
autoconf to detect available libraries, some packages built by this
build daemon may have some disable functionalities.

Bye,
Aurelien

[1]
http://buildd.debian.org/build.php?...563605&file=log

--
.''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
: :' :  Debian developer           | Electrical Engineer
`. `'   aurel32@debian.org         | aurelien@aurel32.net
`-    people.debian.org/~aurel32 | www.aurel32.net


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

Bill:


>A very basic reason is that some packages require 1GB of RAM to build
>in finite time and there are no arm and m68k buildd with that amount of
>RAM.
>
>

Could you try those packages on hedges?  (You can get developer access
from Wookey if you need it).  Hedges has 512MB real and 1.5GB swap.  And
unlike leisner, the netwinders, or nslu2s, it's expandable if needed.

>An alternative is to use distcc+crosscc to a distccd server with 1GB of RAM
.
>
>

I've done this before (rebuilt X that way, just as a test!), but it has
similar concerns to the QEMU approach.

But on that point, I've never had any issues with either
distcc+crossgcc--- which I've tested extensively--- or QEMU.  But
forcing the use of real hardware wherever possible means (a) you know
for sure, and (b) you have to keep your real hardware maintained.  Those
seem like good things.



b.g.

--
Bill Gatliff
bgat@billgatliff.com


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]

Mirco Bauer a écrit :
> Hi Aurelien,
>
> I can give some details about the Mono related packages that failed to
> build before but worked for you. I reply inline.
>
> On Wed, 2006-12-20 at 17:39 +0100, Aurelien Jarno wrote: 

Oops, there is a mistake (copy & paste abuse), this should have been 9
days, as other mono related packages.
[vbcol=seagreen]
> blam failed because of a runtime bug (which is specific to the arm port)
> in Mono, which can be indentified by throwing random
> System.IO.FileNotFoundException, which was fixed in Mono 1.2.2.1 (see
> #394418), so a simple rebuild solves this kind of FTBFS on ARM for Mono
> based software. Though there are remaining problems still with Mono and
> netwinder, which is why the bug isn't closed yet.

When you say "netwinder", could you reproduce it on others netwinder
machines than the build daemon called that way?

FYI this build daemon seems to have some problems (missing files).

--
.''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
: :' :  Debian developer           | Electrical Engineer
`. `'   aurel32@debian.org         | aurelien@aurel32.net
`-    people.debian.org/~aurel32 | www.aurel32.net


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.or
g

[ Post a follow-up to this message ]