Unix Programming - fgets / read / select

Author

fgets / read / select

mdehoon

2005-08-14, 8:49 pm

I am trying to write a function that works like fgets, but calls a hook
function periodically while waiting for a complete line to be ready.
The function looks like this:

char* my_fgets(char buf[], int len, FILE* fp) {
....
fd_set selectset;
int has_input = 0;
int fn = fileno(fp);
while (!has_input) {
struct timeval timeout = {0, 100000}; /* 100000 microseconds
*/
FD_SET(fn, &selectset);
my_hook_function();
has_input = select(fn+1, &selectset, NULL, NULL, &timeout);
}

if (has_input > 0) {
p = fgets(buf, len, fp);
if (p != NULL) return 0; /* No error */
}
....

The "select" function returns as soon as a complete line for reading.

Somebody raised the point though that fgets does buffered IO. So if
"line1\nline2\n" is available for reading from fp, this complete string
will be read into the internal buffer of fgets, and "line1\n\0" is
stored in buf. When my_fgets is then called to read the second line,
the "select" function will think that no input is available, as
"line2\n" was already stored in the internal buffer of fgets. So
my_fgets will fail.

To try this in practice, I wrote the following test program:

int main(void) {
int i;
char buffer[256];
int fn = fileno(stdin);
fd_set selectset;

FD_ZERO(&selectset);
FD_SET(fn, &selectset);
select(fn+1, &selectset, NULL, NULL, NULL);
fgets(buffer, 256, stdin);
printf(buffer);

i = read(fn, buffer, 256);
printf("After the first fgets, %d data are remaining in
stdin\n", i);

FD_ZERO(&selectset);
FD_SET(fn, &selectset);
select(fn+1, &selectset, NULL, NULL, NULL);
fgets(buffer, 256, stdin);
printf(buffer);

return 0;
}

and ran this test program as

mytestprogram < testdata

with the file "testdata" containing "line1\nline2\n".

To my surprise, the second call to "select" does not hang, although the
first call to "fgets" reads all the data in testdata. The program
output is:

$ mytestprogram < testdata
line1
After the first fgets, 0 data are remaining in stdin
line2

I found the same behavior on Linux, Unix, and Cygwin.

So my questions are:
1) Does "select" in some magical way know about the characters that
were already read by fgets and stored in its internal buffers?
2) Is this behavior standard? If it is, my my_fgets function will work
as intended. But I'm worried that this may not work on all systems.
3) Is there some way to do this without relying on fgets? I was
thinking of implementing this with "read", but then I would have to add
some code to break the input into separate lines.

Thanks in advance,

--Michiel.

Floyd L. Davidson

2005-08-15, 2:48 am

"mdehoon" <mjldehoon@yahoo.com> wrote:
>
>and ran this test program as
>
>mytestprogram < testdata
>
>with the file "testdata" containing "line1\nline2\n".
>
>To my surprise, the second call to "select" does not hang, although the
>first call to "fgets" reads all the data in testdata. The program
>output is:
>
>$ mytestprogram < testdata
>line1
>After the first fgets, 0 data are remaining in stdin
>line2

When your program reads data from disk (i.e., not from a tty),
the input mode is "block buffered", and EOF is flagged when the
end of the file is reached. When reading from a tty the mode is
"line buffered" and there is no EOF unless you manually trigger
one by typing a ^D.

If you run /mytestprogram/ without redirecting the input, and
then type in the same data manually, the call to read() will
grab the entire second line, and the second call to select()
will block and wait for a third line... or an EOF (which you
can manually cause with ^D).

When the input is from a file, the entire block of data is
buffered immediately, but since it contains two newlines, it
requires two reads with fgets() to display the data. The second
call to select() returns immediately because EOF has been
flagged on that file descriptor. (Granted the man page for
select() does not tell you that will happen.)

--
Floyd L. Davidson <http://www.apaflo.com/floyd_davidson>
Ukpeagvik (Barrow, Alaska) floyd@apaflo.com

Nils O. Selåsdal

2005-08-17, 7:54 am

mdehoon wrote:
> I am trying to write a function that works like fgets, but calls a hook
> function periodically while waiting for a complete line to be ready.
> The function looks like this:
Mixing select and FILE* gets you in trouble, fast. FILE* usually buffers,
and that won't play well with select, unless you really know what you're
doing :-)