|
Home > Archive > Unix Programming > September 2004 > Reading FIFOs one character at a time
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Reading FIFOs one character at a time
|
|
| Peter Ammon 2004-08-27, 6:21 pm |
| I would like to write a program to read a FIFO (named pipe) one
character per invocation. When I attempt to read the first character,
the writing process finishes its write and terminates, and the first
character is successfully read. However, the remaining data is lost.
(I know this because when I reinvoke the program read() blocks, and when
I write more to the FIFO, I see the newly written data instead of the
old unread data).
I assume what's going on here is that more than one byte is being
buffered in the reading program even though I am using the unbuffered
open() and read() functions, and that extra buffered data is being
discarded when the reading program terminates. How can I prevent this
or otherwise accomplish what I want?
Thanks,
-Peter
| |
| Barry Margolin 2004-08-28, 2:49 am |
| In article <cgoeoe$5hh$1@news.apple.com>,
Peter Ammon <peter_ammon@rocketmail.com> wrote:
> I would like to write a program to read a FIFO (named pipe) one
> character per invocation. When I attempt to read the first character,
> the writing process finishes its write and terminates, and the first
> character is successfully read. However, the remaining data is lost.
>
> (I know this because when I reinvoke the program read() blocks, and when
> I write more to the FIFO, I see the newly written data instead of the
> old unread data).
>
> I assume what's going on here is that more than one byte is being
> buffered in the reading program even though I am using the unbuffered
> open() and read() functions, and that extra buffered data is being
> discarded when the reading program terminates. How can I prevent this
> or otherwise accomplish what I want?
A pipe is a link between two processes, and doesn't really exist unless
there's both a reader and a writer. This is why the first open() blocks
until another process opens the pipe in the other direction, and when
the reader closes its end of the pipe the writer gets a SIGPIPE signal
or EPIPE errno. The pipe doesn't have any permanent storage that
persists when there's no reader.
So you need to ensure that there's always a reader, even when your main
reader process terminates. One way of doing this is to have the writing
process open the pipe with O_RDWR -- it's both a reader and writer. As
long as it never actually calls read() it won't consume anything, so
your repeating reader process will work.
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
| |
| David Schwartz 2004-08-28, 3:18 pm |
|
"Peter Ammon" <peter_ammon@rocketmail.com> wrote in message
news:cgoeoe$5hh$1@news.apple.com...
>I would like to write a program to read a FIFO (named pipe) one character
>per invocation. When I attempt to read the first character, the writing
>process finishes its write and terminates, and the first character is
>successfully read. However, the remaining data is lost.
> (I know this because when I reinvoke the program read() blocks, and when I
> write more to the FIFO, I see the newly written data instead of the old
> unread data).
> I assume what's going on here is that more than one byte is being buffered
> in the reading program even though I am using the unbuffered open() and
> read() functions, and that extra buffered data is being discarded when the
> reading program terminates. How can I prevent this or otherwise
> accomplish what I want?
This should work without any problems. Can you post the shortest
programs needed to reproduce the problem?
DS
| |
| Nils Weller 2004-08-28, 3:18 pm |
| In article <barmar-D77235.23010427082004@comcast.dca.giganews.com>,
Barry Margolin wrote:
> A pipe is a link between two processes, and doesn't really exist unless
> there's both a reader and a writer. This is why the first open() blocks
> until another process opens the pipe in the other direction, and when
> the reader closes its end of the pipe the writer gets a SIGPIPE signal
> or EPIPE errno. The pipe doesn't have any permanent storage that
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^
> persists when there's no reader.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Can you provide a supporting quote from POSIX/UNIX? The close() page of
SUSv3 states:
``When all file descriptors associated with a pipe or FIFO special
file are closed, any data remaining in the pipe or FIFO shall be
discarded.''
I would think that it is OK to close the read end as long as you don't
attempt to perform a write() on the writer file descriptor while there
is no reader (and since the OP's writer program simply exits after
writing its data, this would not be a problem.)
For whatever it's worth, at least FreeBSD 4.8, HP-UX 11i and SuSE 8.2
keep the data in the FIFO if you close the read end and reopen it, but
keep the writer end open all of the time.
--
My real email address is ``nils<at>sipoc<dot>de''
| |
|
| Peter Ammon wrote:
> I would like to write a program to read a FIFO (named pipe) one
> character per invocation. When I attempt to read the first character,
> the writing process finishes its write and terminates, and the first
> character is successfully read. However, the remaining data is lost.
Depending on the OS, a named pipe like this has an internal kernel
buffer. 512 bytes, 4096 bytes, and 32768 bytes I've seen. The OS will
also signal the writer once the reader terminates and thus will bit
bucket the remaining data left in the FIFO.
You've got two problems here, one is the indeterminate FIFO kernel
buffer and the other other is signals between the two processes. I
would suggest you use a message queue a la SysV IPC or some like to
accomplish what you want.
--
JQ
| |
| Norm Dresner 2004-09-02, 6:51 pm |
| "JQ" <foo@bar.com> wrote in message
news:10j7rl0jfk8glc1@corp.supernews.com...
> Peter Ammon wrote:
>
> Depending on the OS, a named pipe like this has an internal kernel
> buffer. 512 bytes, 4096 bytes, and 32768 bytes I've seen. The OS will
> also signal the writer once the reader terminates and thus will bit
> bucket the remaining data left in the FIFO.
>
> You've got two problems here, one is the indeterminate FIFO kernel
> buffer and the other other is signals between the two processes. I
> would suggest you use a message queue a la SysV IPC or some like to
> accomplish what you want.
You can also write an intermediate function to take the whole buffer and
serve up a character at a time.
| |
| Barry Margolin 2004-09-02, 6:51 pm |
| In article
<yD_Yc.536853$Gx4.202351@bgtnsc04-news.ops.worldnet.att.net>,
"Norm Dresner" <ndrez@att.net> wrote:
> "JQ" <foo@bar.com> wrote in message
> news:10j7rl0jfk8glc1@corp.supernews.com...
>
> You can also write an intermediate function to take the whole buffer and
> serve up a character at a time.
Since the reading program terminates after each read, this buffer would
be lost.
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
|
|
|
|
|