Debian Developers - localisation in system wide daemons

Author

localisation in system wide daemons

Nico Golde

2006-11-29, 1:18 pm

Hi,
Adam Cécile reported #400719[0] to the fetchmail package.

The question is wheter a system wide daemon should care
about the system wide locale configurations or not.

Fetchmail currently does, we are not calling it with
LC_MESSAGES=C or something similar.
I can't find anything about this in the policy but to me it
doesnt make sense to use a locale if you dont want it for
some programs.

Since it would be also possible to adjust the settings with
LC_ALL=C in /etc/default/fetchmail I just closed the bug but
reopened it now cause I want to hear some other opinions.
What do you think what is the best way here?
Kind regards
Nico

[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=400719
--
Nico Golde - http://www.ngolde.de
JAB: nion@jabber.ccc.de - GPG: 0x73647CFF
Forget about that mouse with 3/4/5 buttons,
gimme a keyboard with 103/104/105 keys!

Josselin Mouette

2006-11-29, 7:23 pm

Le mercredi 29 novembre 2006 à 19:33 +0100, Nico Golde a écrit :
> Hi,
> Adam Cécile reported #400719[0] to the fetchmail package.
>
> The question is wheter a system wide daemon should care
> about the system wide locale configurations or not.
>
> Fetchmail currently does, we are not calling it with
> LC_MESSAGES=C or something similar.
> I can't find anything about this in the policy but to me it
> doesnt make sense to use a locale if you dont want it for
> some programs.
>
> Since it would be also possible to adjust the settings with
> LC_ALL=C in /etc/default/fetchmail I just closed the bug but
> reopened it now cause I want to hear some other opinions.
> What do you think what is the best way here?

I don't know why exactly fetchmail needs it, but I believe daemons that
have an interface should be localized.
--
Josselin Mouette /\./\

"Do you have any more insane proposals for me?"

Javier Fernández-Sanguino Peña

2006-12-22, 1:34 am

On Wed, Nov 29, 2006 at 07:33:20PM +0100, Nico Golde wrote:
> Hi,
> Adam Cécile reported #400719[0] to the fetchmail package.
>
> The question is wheter a system wide daemon should care
> about the system wide locale configurations or not.

I'd say yes, since if the daemon spits out messages (either error or logs)
the admin will want it to use the system's locale.

BTW, IIRC currently the system's wide locale is set only by the locales
package and introduced in either /etc/default/locale (glibc 2.3.6-5 and
later, which means etch or later) or /etc/environment (woody or earlier) by
it's postinst. Only a few (in my system) init.d scripts read this one in.

> Fetchmail currently does, we are not calling it with
> LC_MESSAGES=C or something similar.

But are you sourcing the system's locale or are you depending on the locale
of the user *starting* fetchmail?

> I can't find anything about this in the policy but to me it
> doesnt make sense to use a locale if you dont want it for
> some programs.

Why would you *not* want a locale? If the program has l10n support and it
provides messages (even in a non-interactive way) there's chances some users
will benefit from the translated messages.

> Since it would be also possible to adjust the settings with
> LC_ALL=C in /etc/default/fetchmail I just closed the bug but
> reopened it now cause I want to hear some other opinions.
> What do you think what is the best way here?

I suggest you introduce a variable in /etc/default/fetchmail named
"USE_SYSTEM_LOCALE" and do that (source the system locale if it exists) if
set to 'Y'. That's better than forcing users to introduce the system locale
in every /etc/default/XXX file and it also makes it easier to switch a
system's locale (no need to touch in many different /etc/default/ files, just
the 'locale' itself). Set the default to whatever you feel comfortable with.

Just my few cents.

Regards

Javier

SZALAY Attila

2006-12-22, 7:31 am

Hi!

On Fri, 2006-12-22 at 03:27 +0100, Javier Fern=C3=A1ndez-Sanguino Pe=C3=B1a=
wrote:
> On Wed, Nov 29, 2006 at 07:33:20PM +0100, Nico Golde wrote:
>=20
> Why would you *not* want a locale? If the program has l10n support and it
> provides messages (even in a non-interactive way) there's chances some us=
ers
> will benefit from the translated messages.

In log files, localized messages may hurt more, than what gain with it.

For example some (semi)automatic log analyzing programs couldn't (and I
think don't want to) handle localized log messages.

Javier Fernández-Sanguino Peña

2006-12-22, 7:31 am

On Fri, Dec 22, 2006 at 08:12:52AM +0100, SZALAY Attila wrote:
>
> In log files, localized messages may hurt more, than what gain with it.
>
> For example some (semi)automatic log analyzing programs couldn't (and I
> think don't want to) handle localized log messages.

IMHO, either that software should be modified to support i18n text or the
admin would have to choose wether he prefers to *understand* the logfile or
to be able to parse it with automatic programs (I believe you are talking
about tools such as logcheck or log-analysis [1][2]).

In any case, it would not be too difficult to adjust programas that parse
logs to be able to parse translated messages. Take in account that all
translated text messages would be available in a message catalog (typicallya
PO file).

So it could be realy straightforward to convert a text mesage like this
(from logcheck's kernel violation.d rules):

^\w{3} [ :0-9]{11} [._[:alnum:]-]+ kernel: [[:alnum:]]+: media error $bad
sector$: status=0x[[:xdigit:]]+ { DriveReady SeekComplete Error }$

to the Spanish equivalent of:

^\w{3} [ :0-9]{11} [._[:alnum:]-]+ kernel: [[:alnum:]]+: error en el medio
$sector defectuoso$: estado=0x[[:xdigit:]]+ { DriveReady SeekComplete Error }$

if the kernel's Spanish PO file has something like:

msgstr "media error (bad sector): status="
msgstr "error en el medio (sector defectuoso): estado="

Or even have logcheck use those PO files directly by introducing some tokens
in its regexps.

For those logparsing programs that would not had i18n support, the user (or
admin) would at least have the *option* to make a decission.

Consider this situation: a user that can not even *read* english (since he
doesn't understand the written language as he uses different script) should
be able to weight which option is more important to him:

a.- be able to use software that generates reports from logfiles with english
messages, and not being able to understand the logfiles themselves and
(probably) not the reports either (if the reporting software is not i18nised)

b.- be able to read the non-english logfiles, but unable to use software to
geenrate reports or summarise logs (until such a software is adapted to
support non-english messages).

What would you chose?

Regards

Javier

[1] There is other more domain-specific log analysis tools (for webservers,
firewalls and mail servers) in Debian but many of that software users
logfiles in a standarised (or propietary) format that is not (typically?
easily?) parseable by humans.

[2] Or Sawmill (but, even if it's a really good and cheap log analysis tool
it still is not free and, consequently, out of our scope)

Gabor Gombas

2006-12-22, 1:17 pm

On Fri, Dec 22, 2006 at 09:37:35AM +0100, Javier Fern=E1ndez-Sanguino Pe=F1=
a wrote:

> In any case, it would not be too difficult to adjust programas that parse
> logs to be able to parse translated messages. Take in account that all
> translated text messages would be available in a message catalog (typical=
ly a
> PO file).

Well, if you already have all messages in a catalog, why don't you
translate the logs on the _viewing_ side?

- No need to bloat the daemon's code
- No need to fear from fresh new locale-related bugs in a lot of
programs running as root
- You do not loose searchability via Google
- If you fire the previous sysadmin who likes to read log files in some
arcane language you still have a chance the next admin can read the
logs too
- In case of network-related messages, you may simply have no other
option since the real error message is generated by a remote process
you have no control over

Btw. googleability IMHO is quite an important reason: most of the time
if you encounter a cryptic log message the best option is to cut&paste
it into a google search and look at the results. With localized log
messages that would never work so you take away the chance from the
poor sysadmin to find out the meaning of the message.

Gabor

--=20
---------------------------------------------------------
MTA SZTAKI Computer and Automation Research Institute
Hungarian Academy of Sciences
---------------------------------------------------------

Gabor Gombas

2006-12-22, 1:17 pm

On Fri, Dec 22, 2006 at 09:37:35AM +0100, Javier Fern=E1ndez-Sanguino Pe=F1=
a wrote:

> b.- be able to read the non-english logfiles,

Btw.: you rarely _read_ logfiles, because most of the time they are just
too terse (unless you already know what they mean, but then it is no
longer important what language do they use). What you do instead is
looking up the message in the documentation (or in Google), and reading
the description _there_. Localized log messages make this a lot harder
to do.

Gabor

--=20
---------------------------------------------------------
MTA SZTAKI Computer and Automation Research Institute
Hungarian Academy of Sciences
---------------------------------------------------------

Nico Golde

2006-12-22, 1:17 pm

Hi,
* Javier Fernández-Sanguino Peña <jfs@computer.org> [2006-12-22 14:43]:
> On Wed, Nov 29, 2006 at 07:33:20PM +0100, Nico Golde wrote:
[...] [vbcol=seagreen]
>
> But are you sourcing the system's locale or are you depending on the locale
> of the user *starting* fetchmail?

No

>
> Why would you *not* want a locale? If the program has l10n support and it
> provides messages (even in a non-interactive way) there's chances some users
> will benefit from the translated messages.

Yes thats also my opinion about it.

>
> I suggest you introduce a variable in /etc/default/fetchmail named
> "USE_SYSTEM_LOCALE" and do that (source the system locale if it exists) if
> set to 'Y'. That's better than forcing users to introduce the system locale
> in every /etc/default/XXX file and it also makes it easier to switch a
> system's locale (no need to touch in many different /etc/default/ files, just
> the 'locale' itself). Set the default to whatever you feel comfortable with.

Thats a good idea even if the parsing in init would be far
more complex. The uncommented LC_ALL=C in fetchmail.default
is just a workaround but could be better.
Thanks!
Nico
--
Nico Golde - http://www.ngolde.de
JAB: nion@jabber.ccc.de - GPG: 0x73647CFF
Forget about that mouse with 3/4/5 buttons,
gimme a keyboard with 103/104/105 keys!

SZALAY Attila

2006-12-23, 1:35 am

Hi All!

On Fri, 2006-12-22 at 09:37 +0100, Javier Fern=C3=A1ndez-Sanguino Pe=C3=B1a=
wrote:
>=20
> IMHO, either that software should be modified to support i18n text or the
> admin would have to choose wether he prefers to *understand* the logfile =
or
> to be able to parse it with automatic programs (I believe you are talking
> about tools such as logcheck or log-analysis [1][2]).=20

Yes, I talk about this programs.

>=20
> So it could be realy straightforward to convert a text mesage like this
> (from logcheck's kernel violation.d rules):

Yes, but if you try to convert all logcheck rule into all language, that
will be a lot more regular expression, and because of this log analyzing
will need a lot more time.=20

> Or even have logcheck use those PO files directly by introducing some tok=
ens
> in its regexps.

I think it's may have problems. For example what about this log message:

syslog(_("This is a log message. problem=3D'%m', severity=3D'%s"),
severity);

What do I do if I want to hide this mesage, if severity lower or equal
to warning?

(I want to say that sometimes the log messages merged from two or more
part)

> For those logparsing programs that would not had i18n support, the user (=
or
> admin) would at least have the *option* to make a decission.=20

I think log messages (which may be sent in network, archived, read by
more than one user, etc.) wouldn't be changed in any circumvent.

Of course it's my opinion.

> Consider this situation: a user that can not even *read* english (since h=
e
> doesn't understand the written language as he uses different script) shou=
ld
> be able to weight which option is more important to him:

And every command name is translated? And every shell command too? I
don't think so.

And some log messages isn't too understandable, even if it's in
someone's native language.

For example in this log message:

2006-12-17T06:41:09+0100 fw ntpd[621]: sendto(148.6.0.1): Bad file descript=
or

the good question is not that how can I translate it to Hungarian,
because it will not help. The good question is, what it cause, and how
can I avoid this.

But tho answer this an expert is more important than the exact meaning
of the message. And as Gabor said for this a stable form of log is very
important.

> a.- be able to use software that generates reports from logfiles with eng=
lish
> messages, and not being able to understand the logfiles themselves and
> (probably) not the reports either (if the reporting software is not i18ni=
sed)
>=20
> b.- be able to read the non-english logfiles, but unable to use software =
to
> geenrate reports or summarise logs (until such a software is adapted to
> support non-english messages).

Hmm. It's a hard question. It's especially hard because I think that to
be a system administrator it's important to know english. And not just
because the log messages, but because the commands and documentations.

And I think an average home user never look into the logs, only if
somebody ask he to do it.

So as a conclusion, I think a.- is my answer.

Michelle Konzack

2007-01-06, 8:05 pm

Hi,

Am 2006-12-22 03:27:43, schrieb Javier Fernández-Sanguino Peña:
>
> Why would you *not* want a locale? If the program has l10n support and it
> provides messages (even in a non-interactive way) there's chances some users
> will benefit from the translated messages.

Maybe the package maintainer do not want to get bugreports
for his/her package in 100 differns languages?

Thanks, Greetings and nice Day
Michelle Konzack
Systemadministrator
Tamay Dogan Network
Debian GNU/Linux Consultant

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
Michelle Konzack Apt. 917 ICQ #328449886
50, rue de Soultz MSM LinuxMichi
0033/6/61925193 67100 Strasbourg/France IRC #Debian (irc.icq.com)

Michelle Konzack

2007-01-06, 8:05 pm

Am 2006-12-22 09:37:35, schrieb Javier Fernández-Sanguino Peña:
> IMHO, either that software should be modified to support i18n text or the
> admin would have to choose wether he prefers to *understand* the logfile or
> to be able to parse it with automatic programs (I believe you are talking
> about tools such as logcheck or log-analysis [1][2]).
>
> In any case, it would not be too difficult to adjust programas that parse
> logs to be able to parse translated messages. Take in account that all
> translated text messages would be available in a message catalog (typically a
> PO file).

Sorry, but "gettext" does not support reverse lookups

> msgstr "media error (bad sector): status="
> msgstr "error en el medio (sector defectuoso): estado="

But you can look-up only for the englisch string and get the translated
back

Now imagine you have 100 *.mo files and each with 100 strings

The loganalyzer must read ALL 100 *.mo files and parse all 100 strings
in it and then compare it with the message in the logfile...

> a.- be able to use software that generates reports from logfiles with english
> messages, and not being able to understand the logfiles themselves and
> (probably) not the reports either (if the reporting software is not i18nised)

If a non-native english speaker want to analyze logfiles, the loganalyzer
should do the translation...

The log analyzer should be configurable if it get a string like

----8<--------------------------------------------------------------
Dec 27 12:25:01 michelle1 init: Re-reading inittab
Dec 27 12:25:01 michelle1 last message repeated 2 times
Dec 27 12:28:08 michelle1 uptimed: milestone: 9 days, 00:00:00 (Neun Tage)
Dec 27 12:28:08 michelle1 sSMTP[11044]: Unable to set AuthUser="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX@tamay-dogan.net"
Dec 27 12:28:08 michelle1 sSMTP[11044]: Unable to set AuthPass="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
Dec 27 12:28:08 michelle1 sSMTP[11044]: Server didn't accept AUTH LOGIN (535 Authentication rejected)
Dec 27 12:30:01 michelle1 init: Re-reading inittab
Dec 27 12:30:01 michelle1 last message repeated 2 times
<snip>
Dec 27 21:20:01 michelle1 init: Re-reading inittab
Dec 27 21:20:01 michelle1 last message repeated 2 times
Dec 27 21:21:23 michelle1 rpc.statd[724]: Received erroneous SM_UNMON request from michelle1.private for 192.168.0.69
Dec 27 21:25:03 michelle1 init: Re-reading inittab
Dec 27 21:25:03 michelle1 last message repeated 2 times
<snip>
Dec 28 00:30:25 michelle1 /usr/sbin/gpm[664]: select(): Interrupted system call
----8<--------------------------------------------------------------

So if the loganalyzer read " sSMTP[" it should use the "ssmtp.mo"
file and if it read " rpc.statd" from "nfs-common.mo".

I have seenm there is apossibility to get all english strings from
an mo file so if you have a message like

Dec 27 12:28:08 michelle1 sSMTP[11044]: Unable to set AuthUser="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX@tamay-dogan.net"

the loganalyzer can cut it to

sSMTP[11044]: Unable to set AuthUser="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX@tamay-dogan.net"

detect the "sSMTP" program and do a Fuzzy-Search in the ssmtp.mo for
"Unable to set " which will probably work since the original String
seems to be

msgid "Unable to set %s=\"%s\"\n"

if it would be localized.

> What would you chose?
>
> Regards
>
> Javier

Thanks, Greetings and nice Day
Michelle Konzack
Systemadministrator
Tamay Dogan Network
Debian GNU/Linux Consultant

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
Michelle Konzack Apt. 917 ICQ #328449886
50, rue de Soultz MSM LinuxMichi
0033/6/61925193 67100 Strasbourg/France IRC #Debian (irc.icq.com)

Marc Haber

2007-01-06, 8:05 pm

On Sat, 30 Dec 2006 19:13:57 +0100, Michelle Konzack
<linux4michelle@freenet.de> wrote:
>Maybe the package maintainer do not want to get bugreports
>for his/her package in 100 differns languages?

You are - again - taking about something you don't understand.

=46or the record: I maintain two widely installed packages which are
translated to a gazillion of languages (exim4 and adduser, to be
exact), and I have never received a non-english bug report.

The "worst" that happened was that somebody contacted my via private
e-mail in German, but these somebodies probably knew that German is my
native language.

Oh, yeah, right. Kindly fix your signature. It is ridiculously
overlong.

Greetings
Marc

--=20
-------------------------------------- !! No courtesy copies, please !! =
-----
Marc Haber | " Questions are the | Mailadresse im =
Header
Mannheim, Germany | Beginning of Wisdom " | =
http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 =
72739834

Szalay Attila

2007-01-06, 8:05 pm

Hi All!

On v, 2006-12-31 at 00:08 +0100, Marc Haber wrote:
>
> For the record: I maintain two widely installed packages which are
> translated to a gazillion of languages (exim4 and adduser, to be
> exact), and I have never received a non-english bug report.

Then you lucky or your packages do it right.

My problem is not with the bug report language but the cut&pasted log
messages language. If I got a bug report I allways ask for log messages
(not for my packages but in some mailing list) and it's vwey confusable
if I cannot read or understand the log messages. And if I had have a
package with a lot of messages it would have been a PIA. (IMHO)

PS: My signature is correct or too short?

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Marc Haber

2007-01-06, 8:05 pm

On Sun, 31 Dec 2006 08:48:50 +0100, Szalay Attila <sasa@debian.org>
wrote:
>My problem is not with the bug report language but the cut&pasted log
>messages language.

I have received some bug reports which contain logs from a non-english
locale, but in the cases that I have not been able to figure it out I
have asked the submitter to retry with LANG=3DC.

Actually no big deal here.

Greetings
Marc

--=20
-------------------------------------- !! No courtesy copies, please !! =
-----
Marc Haber | " Questions are the | Mailadresse im =
Header
Mannheim, Germany | Beginning of Wisdom " | =
http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 =
72739834

Gabor Gombas

2007-01-06, 8:06 pm

On Sun, Dec 31, 2006 at 12:15:27PM +0100, Marc Haber wrote:

> I have received some bug reports which contain logs from a non-english
> locale, but in the cases that I have not been able to figure it out I
> have asked the submitter to retry with LANG=C.
>
> Actually no big deal here.

If the log message talks about some data corruption and the user wants
to ask how to salvage the data then answering "please reproduce the
exact corruption on an English-speaking system" sounds problematic...

Gabor

--
---------------------------------------------------------
MTA SZTAKI Computer and Automation Research Institute
Hungarian Academy of Sciences
---------------------------------------------------------

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Nico Golde

2007-01-06, 8:06 pm

Hi,
* Gabor Gombas <gombasg@sztaki.hu> [2006-12-31 14:54]:
> On Sun, Dec 31, 2006 at 12:15:27PM +0100, Marc Haber wrote:
>
>
> If the log message talks about some data corruption and the user wants
> to ask how to salvage the data then answering "please reproduce the
> exact corruption on an English-speaking system" sounds problematic...

Shouldnt be that big kind of a problem to look in the .po
file for the english message in such a corner case.
Kind regards
Nico
--
Nico Golde - http://www.ngolde.de
JAB: nion@jabber.ccc.de - GPG: 0x73647CFF
Forget about that mouse with 3/4/5 buttons,
gimme a keyboard with 103/104/105 keys!

Michelle Konzack

2007-01-11, 1:30 pm

Hello all,

sorry for the late answer but I am fighting with a SPAM bomb...

Am 2006-12-31 08:48:50, schrieb Szalay Attila:
> Hi All!

> Then you lucky or your packages do it right.

>
> My problem is not with the bug report language but the cut&pasted log
> messages language. If I got a bug report I allways ask for log messages
> (not for my packages but in some mailing list) and it's vwey confusable
> if I cannot read or understand the log messages. And if I had have a
> package with a lot of messages it would have been a PIA. (IMHO)

I have customers which use byside Debian some another distributions
and it is realy annoying, if th init scripts output localized
messages and they apear localized in the logfiles.

I have two RedHat systems to maintain which do that and the messages
are in french.

I do not like to see Debian doing the same thing...

Thanks, Greetings and nice Day
Michelle Konzack

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
Michelle Konzack Apt. 917 ICQ #328449886
50, rue de Soultz MSM LinuxMichi
0033/6/61925193 67100 Strasbourg/France IRC #Debian (irc.icq.com)

Don Armstrong

2007-01-11, 1:30 pm

On Thu, 11 Jan 2007, Michelle Konzack wrote:
> Am 2006-12-31 08:48:50, schrieb Szalay Attila:
>
>
> I have customers which use byside Debian some another distributions
> and it is realy annoying, if th init scripts output localized
> messages and they apear localized in the logfiles.

Outputting messages in the systemwide locale is always appropriate. If
you do not want them localized (whatever language that means changing
them into) then the system wide locale should not be set to that
language.

It's trivial to change the language to C if that's a problem.

Don Armstrong

--
"A one-question geek test. If you get the joke, you're a geek: Seen on
a California license plate on a VW Beetle: 'FEATURE'..."
-- Joshua D. Wachs - Natural Intelligence, Inc.

http://www.donarmstrong.com http://rzlab.ucr.edu

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Christian Perrier

2007-01-12, 7:24 am

> I do not like to see Debian doing the same thing...

You mean doing what's needed to be by default accessible to people who
don't speak English if they did choose to install in another language
than English?

Could you explain us your perception of the harm caused by this?

Wouter Verhelst

2007-01-15, 1:19 pm

On Fri, Jan 12, 2007 at 07:16:12AM +0100, Christian Perrier wrote:
>
>
> You mean doing what's needed to be by default accessible to people who
> don't speak English if they did choose to install in another language
> than English?
>
> Could you explain us your perception of the harm caused by this?

There are a number of log message parsers that work based on regular
expressions for the entries in these log files. These will stop working
properly, for no immediately ascertainable reason. Providing localized
regular expressions would not only be the worst possible workaround; it
would also be the only one I can come up with.

While I agree that efforts to translate the system at all levels are
laudable, I'm not sure it's a good thing to break useful software for
the sake of said translation efforts. And yes, log file parsers are
useful programs, not just on servers, and even if you do want the rest
of your system to appear localized.

At the very least, if we're going to translate log messages, then there
should be an easy switch to disable such translation for log messages
alone (while not for the rest of the system).

--
<Lo-lan-do> Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Andrea Bolognani

2007-01-15, 1:19 pm

On Mon, 15 Jan 2007 18:38:29 +0100
Wouter Verhelst <wouter@debian.org> wrote:

> At the very least, if we're going to translate log messages, then there
> should be an easy switch to disable such translation for log messages
> alone (while not for the rest of the system).

If we want to provide our non-English-speaking users with localized log
messages, wouldn't it be possible to create a program which translates the
logs on the fly, based on the message catalog we should write anyway?

Doing so, we won't break any existing log analyzer.

--
KiyuKo <eof AT kiyuko DOT org>
Resistance is futile, you will be garbage collected.

Simon Richter

2007-01-16, 1:31 am

Hello,

Andrea Bolognani wrote:

> If we want to provide our non-English-speaking users with localized log
> messages, wouldn't it be possible to create a program which translates the
> logs on the fly, based on the message catalog we should write anyway?

IMO, the proper solution would be machine-readable log information,
probably similar to lintian output ("dash-separated-tag variable1
variable2"), that can be postprocessed into a human-readable text in the
appropriate language or analysed by automated tools.

Writing localised strings into a logfile makes it rather difficult for
people to write proper bug reports.

Simon

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Kevin Mark

2007-01-16, 1:31 am

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, Jan 15, 2007 at 06:54:24PM +0100, Andrea Bolognani wrote:
> On Mon, 15 Jan 2007 18:38:29 +0100
> Wouter Verhelst <wouter@debian.org> wrote:
>
>
> If we want to provide our non-English-speaking users with localized log
> messages, wouldn't it be possible to create a program which translates the
> logs on the fly, based on the message catalog we should write anyway?

data flow:
A->B->-C->D

A=program that logs (cpu temp monitor for FAN #1, FAN #2,...)
B=data that is logged (current temp data set like [30,40,..])
C=formating "FAN #%d is %d"
D=current log file "FAN #1 is 30c\nFAN #2 is 40c\n..."

To translate D is very hard but that seems what most programs output.

So D needs to exists for backwards compatibility.

If B is also created as a 'data' log file, then a program can be created
to use localization base upon C to produce localized logs.
I suppose that a program can use a 'tee' to produce 2 logs, one standard
and the other localized.

A->B-C->D (the standard english one)
|
+->E (the localized one)

But then there is an issue about how to solve problems based upon log
messages. A localized message can help the user to understand the problem,
but then if used to search say google, should they use the standard
english one or the localized one? 100% of google results will find the
english one currently. If more than one languages is used, the ability
to find results is signfigantly reduced.
- -K
[disregard if not useful, flames not appreciated]
- --
| .''`. == Debian GNU/Linux == | my web site: |
| : :' : The Universal | 'under construction' |
| `. `' Operating System | go to counter.li.org and |
| `- http://www.debian.org/ | be counted! #238656 |
| my keysever: subkeys.pgp.net | my NPO: cfsg.org |
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFFrFDyv8UcC1qRZVMRAl3qAJ91I1tgeb/uDb3KM53of9quiLPX2wCfbPre
whgAiqFMuScNWdRCZ/qBjXQ=
=6Kyy
-----END PGP SIGNATURE-----

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Michelle Konzack

2007-01-16, 1:19 pm

Am 2007-01-12 07:16:12, schrieb Christian Perrier:
>
>
> You mean doing what's needed to be by default accessible to people who
> don't speak English if they did choose to install in another language
> than English?
>
> Could you explain us your perception of the harm caused by this?

This peoples will ask you for help and send all
the logfiles maybe in arabic, hindu or africaan.

Thanks, Greetings and nice Day
Michelle Konzack

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
Michelle Konzack Apt. 917 ICQ #328449886
50, rue de Soultz MSM LinuxMichi
0033/6/61925193 67100 Strasbourg/France IRC #Debian (irc.icq.com)

Don Armstrong

2007-01-16, 7:28 pm

On Tue, 16 Jan 2007, Michelle Konzack wrote:
> Am 2007-01-12 07:16:12, schrieb Christian Perrier:
>
> This peoples will ask you for help and send all the logfiles maybe
> in arabic, hindu or africaan.

So you ask them to translate or rerun the program with an appropriate
locale. Being able to speak all of the languages that Debian is in
isn't a requirement to help anymore than knowing english should be[1]
a requirement to administer one's own system.

Don Armstrong

1: Even though it unfortunatly almost is...
--
"I was thinking seven figures," he said, "but I would have taken a
hundred grand. I'm not a greedy person." [All for a moldy bottle of
tropicana.]
-- Sammi Hadzovic [in Andy Newman's 2003/02/14 NYT article.]
http://www.nytimes.com/2003/02/14/nyregion/14EYEB.html

http://www.donarmstrong.com http://rzlab.ucr.edu

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Jean-Christophe Dubacq

2007-01-16, 7:28 pm

Le 16 janv. 07 =E0 20:13, Don Armstrong a =E9crit :

>
> So you ask them to translate or rerun the program with an appropriate
> locale. Being able to speak all of the languages that Debian is in
> isn't a requirement to help anymore than knowing english should be[1]
> a requirement to administer one's own system.

Unless the bug is sporadic, happens only in some other locale...

I think log output should be reparsable enough, and interfaces (other =20=

than less) can translate the logs when viewing them (let's say gnome-=20
system-log-viewer...).
--=20
JCD

Manoj Srivastava

2007-01-18, 7:30 am

On Tue, 16 Jan 2007 17:15:50 +0100, Michelle Konzack <linux4michelle@freenet.de> said:

> Am 2007-01-12 07:16:12, schrieb Christian Perrier:
[vbcol=seagreen]
> This peoples will ask you for help and send all the logfiles maybe
> in arabic, hindu or africaan.

Hindu is an adjective that describes someone whose faith is
hinduism. It is not a language.

manoj
--
Drop that pickle!
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Felipe Sateler

2007-01-18, 7:23 pm

Manoj Srivastava wrote:
>
> Hindu is an adjective that describes someone whose faith is
> hinduism. It is not a language.
>
> manoj

And, according to Merriam-Webster[1], a native or inhabitant from India.

[1] http://www.m-w.com/cgi-bin/dictiona...onary&va=hindoo

--

Felipe Sateler

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Manoj Srivastava

2007-01-19, 1:18 pm

On Thu, 18 Jan 2007 19:05:41 -0300, Felipe Sateler <fsateler@gmail.com> said:

> Manoj Srivastava wrote:
[vbcol=seagreen]
> And, according to Merriam-Webster[1], a native or inhabitant from
> India.

> [1] http://www.m-w.com/cgi-bin/dictiona...onary&va=hindoo

Merriam-Webster is falling into the old non-PC
over-generalization that all inhabitants of India are Hindus. I
assure you, the 30% Muslim population and 10% Christian population
would not be pleased at being referred to as "Hindus". Please don't
do it, unless your intent is to be offensive.

manoj
--
The chief enemy of creativity is "good" sense Picasso
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Russ Allbery

2007-01-19, 1:18 pm

Felipe Sateler <fsateler@gmail.com> writes:
> Manoj Srivastava wrote:

[vbcol=seagreen]
> And, according to Merriam-Webster[1], a native or inhabitant from India.

> [1] http://www.m-w.com/cgi-bin/dictiona...onary&va=hindoo

There are scattered bits of imperialism in the English language that went
in when the people speaking it didn't care about fine distinctions, such
as the religions of people they were conquering or the difference between
a religion and an ethnicity. It's probably best to not preserve them.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Steve Langasek

2007-01-21, 1:26 am

On Tue, Jan 16, 2007 at 11:13:58AM -0800, Don Armstrong wrote:
> On Tue, 16 Jan 2007, Michelle Konzack wrote:
[vbcol=seagreen]
[vbcol=seagreen]
> So you ask them to translate or rerun the program with an appropriate
> locale. Being able to speak all of the languages that Debian is in
> isn't a requirement to help anymore than knowing english should be[1]
> a requirement to administer one's own system.

Translations are not necessarily reversible; this depends heavily on both
the quality of the translation and the quality of the original messages.
And logged errors are not necessarily reproducible on demand.

AFAICS, post-processing of log messages would be the most reliable method to
give admins localized logs while also making it feasible for upstreams to
support user requests. Any problems that would make it hard to post-process
English logs for localization would apply n-fold to post-processing
non-English logs for translation back to English.

For best results, we would have a logging protocol that logs a message ID
plus arguments, so that formatting into English /or/ into other languages
would follow the same, sscanf-free process.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
vorlon@debian.org http://www.debian.org/

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Don Armstrong

2007-01-21, 1:26 am

On Sat, 20 Jan 2007, Steve Langasek wrote:
> AFAICS, post-processing of log messages would be the most reliable
> method to give admins localized logs while also making it feasible
> for upstreams to support user requests. Any problems that would make
> it hard to post-process English logs for localization would apply
> n-fold to post-processing non-English logs for translation back to
> English.

But this style of post-processing would occur only when there was a
support request that required an english speaker to look at the logs;
the user translating it by hand in this case (or using an appropriate
locale, or finding someone who could translate it or backtracking from
the message to english using the same mapping that the program does)
seems perfectly reasonable to me.

Post-processing seems to require a set of fragile dependencies between
the log processing software and the actual software generating the
messages unless someone standardizes on a central repository of
messages in different languages [and would make casual log checking
slightly more difficult.]

> For best results, we would have a logging protocol that logs a
> message ID plus arguments, so that formatting into English /or/ into
> other languages would follow the same, sscanf-free process.

And a central repository of all of the message Ids and blocks which
are assigned to speciifc programs and whatever other standards are
needed to implement it... I suppose it would be optimal, but I don't
think it'll happen anytime soon.

Don Armstrong

--=20
[A] theory is falsifiable [(and therefore scientific) only] if the
class of its potential falsifiers is not empty.
-- Sir Karl Popper _The Logic of Scientific Discovery_ =A721

http://www.donarmstrong.com http://rzlab.ucr.edu

Steve Langasek

2007-01-21, 7:25 am

On Sat, Jan 20, 2007 at 10:45:07PM -0800, Don Armstrong wrote:
> On Sat, 20 Jan 2007, Steve Langasek wrote:
[vbcol=seagreen]
> But this style of post-processing would occur only when there was a
> support request that required an english speaker to look at the logs;
> the user translating it by hand in this case (or using an appropriate
> locale, or finding someone who could translate it or backtracking from
> the message to english using the same mapping that the program does)
> seems perfectly reasonable to me.

But translations aren't guaranteed to be reversible, even in the context of
the limited set of messages used by a particular program.

> Post-processing seems to require a set of fragile dependencies between
> the log processing software and the actual software generating the
> messages unless someone standardizes on a central repository of
> messages in different languages [and would make casual log checking
> slightly more difficult.]

If implemented right, checking the post-processed localized logs *would* be
casual.

[vbcol=seagreen]
> And a central repository of all of the message Ids and blocks which
> are assigned to speciifc programs and whatever other standards are
> needed to implement it... I suppose it would be optimal, but I don't
> think it'll happen anytime soon.

Um, this wouldn't need to be any more centralized than existing gettext
handling is today.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
vorlon@debian.org http://www.debian.org/

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Miles Bader

2007-01-22, 1:30 am

Manoj Srivastava <srivasta@debian.org> writes:
> Merriam-Webster is falling into the old non-PC
> over-generalization that all inhabitants of India are Hindus.

"Merriam-Webster" isn't falling into anything, it's a dictionary, and
dictionaries describe usage.

I know from reading old books that this particular usage (usually with
the slightly odd spelling given "hindoo") used to be very common. I
presume that most people these days are better informed, so perhaps
there should be some notation in the dictionary like "archaic" or
whatever, but I don't know if they generally make such distinctions.

-Miles

--
We have met the enemy, and he is us. -- Pogo

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Manoj Srivastava

2007-01-22, 1:30 am

On Mon, 22 Jan 2007 12:21:47 +0900, Miles Bader <miles.bader@necel.com> said:

> Manoj Srivastava <srivasta@debian.org> writes:
[vbcol=seagreen]
> "Merriam-Webster" isn't falling into anything, it's a dictionary,
> and dictionaries describe usage.

I hate to break it to you (and I do sincerely apologize for
spoiling your innocence), but lexicons are not handed down to mankind
on tablets writ in stone as some kind of divine donation.

Dictionaries, and, gasp, encyclopedias, have been known to
have factual errors, and being written by humans, can actually have
old non-PC over-generalizations due to the biases of the editors
writing the darned book.

Seeing that I do not believe in the divinity or the
infallibility of dictionaries, I stand by my statement: considering
that the etymology of the word is from Hindi, I consider myself has
someone with a modicum if knowledge on the subject.

manoj
--
The heart is wiser than the intellect.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Miles Bader

2007-01-22, 1:30 am

Manoj Srivastava <srivasta@debian.org> writes:
> I hate to break it to you (and I do sincerely apologize for
> spoiling your innocence), but lexicons are not handed down to mankind
> on tablets writ in stone as some kind of divine donation.

I was not claiming that.

Indeed, I was claiming exactly the opposite, that the dictionary was
describing a _usage_ of the word by english speakers. I've personally
seen many such uses with exactly the meaning given, so it's pretty clear
to me that the dictionary has not somehow screwed up in that mission.

As I said, it's more or less archaic usage these days, and it would be
nice if they indicated that, but I do not know how dictionaries handle
such cases in general.

-Miles

--
Ich bin ein Virus. Mach' mit und kopiere mich in Deine .signature.

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Manoj Srivastava

2007-01-22, 1:30 am

On Mon, 22 Jan 2007 13:19:24 +0900, Miles Bader <miles.bader@necel.com> said:

> Manoj Srivastava <srivasta@debian.org> writes:
[vbcol=seagreen]
> I was not claiming that.

> Indeed, I was claiming exactly the opposite, that the dictionary was
> describing a _usage_ of the word by english speakers. I've
> personally seen many such uses with exactly the meaning given, so
> it's pretty clear to me that the dictionary has not somehow screwed
> up in that mission.

The etymology of the word is not english: (Hindu + stan). When
incorporating it into English, the word came with a well defined
meaning.

> As I said, it's more or less archaic usage these days, and it would
> be nice if they indicated that, but I do not know how dictionaries
> handle such cases in general.

So: I said three things about the entry:
i) Old: you have yourself agreed that it is archaic, which, correct
me if I am wrong, does indeed imply old.
ii) Non PC; Trust me, labelling muslims, sikhs, budhists, and
christians "Hindus" is going to get you some very upset people,
and indeed, can be slightly dangerous. Mislabelling people's
sreligious beliefs can be graught with danger (consider what an
assertion someone followed judaism would have had in europe, oh,
some 70 years ago).
So: Definiteky non PC
iii) Over generalization: the people living in India are not, by a
long shot, followers of Hinduism, despite the easy imperialist
assumptions. While Hinduism is a majority religion, it is not
the only one.

I said, and I said again, and thrice I say and done: the
dictionary in question is using an old, non-PC, over generalization;
and people who use it as such are bound to insult the people they are
so referring.

Y'all are free to insult all non-hindu inhabitants of India,
but he who sows the wind ...

manoj
--
Good-bye. I am leaving because I am bored. George Saunders' dying
words
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Miles Bader

2007-01-22, 1:30 am

Manoj Srivastava <srivasta@debian.org> writes:
> The etymology of the word is not english: (Hindu + stan). When
> incorporating it into English, the word came with a well defined
> meaning.

Apparently the english authors who used it didn't realize that, and I
assume their usage reflects the general usage at the time they were
writing. Anyway, for whatever reason, the given usage was present, and
reasonably common, _in english_ at one time. The dictionary documents
that, as is their job.

-miles
--
Any man who is a triangle, has thee right, when in Cartesian Space, to
have angles, which when summed, come to know more, nor no less, than
nine score degrees, should he so wish. [TEMPLE OV THEE LEMUR]

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

David Moreno Garza

2007-01-22, 7:21 am

On Fri, 2007-01-19 at 10:36 -0800, Russ Allbery wrote:
> There are scattered bits of imperialism in the English language that went
> in when the people speaking it didn't care about fine distinctions, such
> as the religions of people they were conquering or the difference between
> a religion and an ethnicity. It's probably best to not preserve them.

Right. Or the way a continent and a country is called.

David.

Manoj Srivastava

2007-01-22, 1:17 pm

On Mon, 22 Jan 2007 15:01:25 +0900, Miles Bader <miles.bader@necel.com> said:

> Manoj Srivastava <srivasta@debian.org> writes:
[vbcol=seagreen]
> Apparently the english authors who used it didn't realize that, and
> I assume their usage reflects the general usage at the time they
> were writing. Anyway, for whatever reason, the given usage was
> present, and reasonably common, _in english_ at one time. The
> dictionary documents that, as is their job.

I think we have some reading comprehension issues here. Let me
try one more time. The usage you speak of is:
i) Old
ii) Non PC
iii) An over generalization

If the dictionary is still perpetuating that definition, it is
falling into a pattern of disseminating information that is:
i) Old
ii) Non PC
iii) An over generalization

HTH. HAND,

manoj
--
strategy, n.: A comprehensive plan of inaction.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Michelle Konzack

2007-01-25, 7:26 pm

Am 2007-01-18 01:28:34, schrieb Manoj Srivastava:
> On Tue, 16 Jan 2007 17:15:50 +0100, Michelle Konzack <linux4michelle@freenet.de> said:
>
> Hindu is an adjective that describes someone whose faith is
> hinduism. It is not a language.

OK 1:0 for you... the language name is Hindi.

Hey, how do you know that?

Thanks, Greetings and nice Day
Michelle Konzack

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
Michelle Konzack Apt. 917 ICQ #328449886
50, rue de Soultz MSM LinuxMichi
0033/6/61925193 67100 Strasbourg/France IRC #Debian (irc.icq.com)

Wouter Verhelst

2007-01-26, 7:21 am

On Sat, Jan 20, 2007 at 10:45:07PM -0800, Don Armstrong wrote:
> On Sat, 20 Jan 2007, Steve Langasek wrote:
>
> But this style of post-processing would occur only when there was a
> support request that required an english speaker to look at the logs;

No; it would also occur when there is an automated system that tries to
look at the logs in an attempt to find something useful.

I've been thinking that it's not too hard to do this. The syslog()
API looks like this:

void openlog(const char *ident, int option, int facility);
void syslog(int priority, const char *format, ...);
void closelog(void);

(ignoring vsyslog for now, which takes a va_list)

Provided people are happy with the assumption that the "ident" argument
that's being presented to openlog() is valid as a gettext-style
domainname, all that's needed is a syslogd which is slightly smarter
than a regular syslogd, and which writes out messages it gets from the
syslog() call to two files: once translated, once not. Or, perhaps even
better: once translated, once not, and once without even the
printf-style processing that normally occurs in the syslog() call.

> Post-processing seems to require a set of fragile dependencies between
> the log processing software and the actual software generating the
> messages unless someone standardizes on a central repository of
> messages in different languages [and would make casual log checking
> slightly more difficult.]

Not quite.

--
<Lo-lan-do> Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22

--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Stephen Gran

2007-01-26, 7:21 am

This one time, at band camp, Wouter Verhelst said:
> On Sat, Jan 20, 2007 at 10:45:07PM -0800, Don Armstrong wrote:
>
> No; it would also occur when there is an automated system that tries to
> look at the logs in an attempt to find something useful.
>
> I've been thinking that it's not too hard to do this. The syslog()
> API looks like this:

It also occurs to me that you could just set something in syslog.conf
that pipes to a wrapper script that translates for you.

*.* | translate-logs

or something. I haven't tried this in practice, but it seems like it
would work with the minimum of changes.
--
-----------------------------------------------------------------
| ,''`. Stephen Gran |
| : :' : sgran@debian.org |
| `. `' Debian user, admin, and developer |
| `- http://www.debian.org |
-----------------------------------------------------------------