I am reading a large text file a chunk at a time using the
readlines(buffer_size) statement. I get an IOERROR ERRNO 32 Broken
Pipe command when I "pipe" the output to mysql database using the
following command:

python script.py | mysql

I am using RED HAT 9 Linux. The problem only happens if I use the pipe
to MYSQL. It breaks but it gets to process around 1000 MYSQL
statements.

If I only use the following statement:

python script.py

The programs runs ok and reads all text records without problems.  I
do need to sent the pipe to mysql so that I can populate a database.

I tried using the following statement but it still breaks:

import signal

signal.signal(signal.SIGPIPE, signal.SIG_DFL)

Thanks,

Ruben

[ Post a follow-up to this message ]

Ruben <fernandezvictor77@hotmail.com> wrote:
> I am reading a large text file a chunk at a time using the
> readlines(buffer_size) statement. I get an IOERROR ERRNO 32 Broken
> Pipe command when I "pipe" the output to mysql database using the
> following command:

> Python script.py | mysql

> I am using RED HAT 9 Linux. The problem only happens if I use the pipe
> to MYSQL. It breaks but it gets to process around 1000 MYSQL
> statements.

The SIGPIPE signal is received by the writing process if there's
suddenly no reader anymore. So it looks as if your 'mysql' process
dies prematurely for some reason. Perhaps you should out the 'tee'
command in between to be able to see what really gets send to 'mysql'.

> If I only use the following statement:

> Python script.py

> The programs runs ok and reads all text records without problems.  I
> do need to sent the pipe to mysql so that I can populate a database.

> I tried using the following statement but it still breaks:

>    import signal
>    signal.signal(signal.SIGPIPE, signal.SIG_DFL)

The default action for SIGPIPE is to kill the process that receives
it. But even if you would manage to keep the process alive by
giving the signal some other disposition that still wouldn't help
when the other side you want to send commands to does not live
anymore...
Regards, Jens
--
\   Jens Thoms Toerring  ___  Jens.Toerring@physik.fu-berlin.de
\__________________________  http://www.toerring.de

[ Post a follow-up to this message ]