Unix Programming - read() on a regular file

Author

read() on a regular file

Henry Townsend

2006-09-10, 1:21 pm

Is there any circumstance in which read() on a regular file will *not*
be able to transfer the entire file in one go? I realize there are many
complications with pipes, sockets, terminals, etc but what about files?
Say I have a regular file and open() it, fstat() to get the size, then
malloc that many bytes (assume it's not too big to buffer in memory).
Can I then issue a single

read(fd, buf, size);

and be confident it will transfer <size> bytes?

The documentation strongly implies this but I do not see a promise anywhere.

Thanks,
HT

wwwdev

2006-09-10, 6:08 pm

Hi Henry Townsend,

another process can change file size between your fstat() and read().

Rainer Weikusat

2006-09-11, 1:32 am

Henry Townsend <henry.townsend@not.here> writes:
> Is there any circumstance in which read() on a regular file will *not*
> be able to transfer the entire file in one go? I realize there are
> many complications with pipes, sockets, terminals, etc but what about
> files? Say I have a regular file and open() it, fstat() to get the
> size, then malloc that many bytes (assume it's not too big to buffer
> in memory). Can I then issue a single
>
> read(fd, buf, size);
>
> and be confident it will transfer <size> bytes?

Smaller amounts of data may be transferred if somebody truncates the
file between fstat and read or if the operation was interrupted by a
signal. The latter is especially nasty if you are on Linux and use any
of the 'usual' Linux pthread-librariers, which all cause seemingly
innocent syscalls to return EINTR, no matter if the application itself
is handling any signals or not.

Paul Pluzhnikov

2006-09-11, 1:32 am

Rainer Weikusat <rainer.weikusat@sncag.com> writes:

> Henry Townsend <henry.townsend@not.here> writes:
>
> Smaller amounts of data may be transferred if somebody truncates the
> file between fstat and read or if the operation was interrupted by a
> signal.

Also consider unreadable disk blocks (failing disk).

I am not sure whether you'll get a partial read, or no data at all
(probably depends on the OS).

Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.

Barry Margolin

2006-09-11, 1:32 am

In article <m3odtnbl15.fsf@somewhere.in.california.localhost>,
Paul Pluzhnikov <ppluzhnikov-nsp@charter.net> wrote:

> Rainer Weikusat <rainer.weikusat@sncag.com> writes:
>
>
> Also consider unreadable disk blocks (failing disk).
>
> I am not sure whether you'll get a partial read, or no data at all
> (probably depends on the OS).

I would expect no data and an error in that case. Disk errors are
considered very serious.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

Lew Pitcher

2006-09-11, 1:32 am

Henry Townsend wrote:
> Is there any circumstance in which read() on a regular file will *not*
> be able to transfer the entire file in one go? I realize there are many
> complications with pipes, sockets, terminals, etc but what about files?
> Say I have a regular file and open() it, fstat() to get the size, then
> malloc that many bytes (assume it's not too big to buffer in memory).
> Can I then issue a single
>
> read(fd, buf, size);
>
> and be confident it will transfer <size> bytes?

There are circumstances where such an organization will not transfer
all of the requested data. Primarily, anything that can interrupt such
a transfer (such as a timer pop or other 'caught' async signal) can
cause the read() to terminate "early", returning less data than
requested.

Quote:
"On success, the number of bytes read is returned (zero
indicates end of file), and the file position is advanced
by this number. It is not an error if this number is
smaller than the number of bytes requested; this may hap=AD
pen for example because fewer bytes are actually available
right now (maybe because we were close to end-of-file, or
because we are reading from a pipe, or from a terminal),
or because read() was interrupted by a signal."
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

There are other conditions that cause read() to act similarly,
primarily I/O errors and problems with data not being ready on a
non-blocking read.

HTH
--=20
Lew Pitcher

Nils O. Selåsdal

2006-09-11, 1:32 am

Henry Townsend wrote:
> Is there any circumstance in which read() on a regular file will *not*
> be able to transfer the entire file in one go? I realize there are many
> complications with pipes, sockets, terminals, etc but what about files?
> Say I have a regular file and open() it, fstat() to get the size, then
> malloc that many bytes (assume it's not too big to buffer in memory).
> Can I then issue a single
>
> read(fd, buf, size);
>
> and be confident it will transfer <size> bytes?
>
> The documentation strongly implies this but I do not see a promise
> anywhere.
This is from the UNP book, use that so you don't need to speculate how
much read will read under certain circumstances..

ssize_t /* Read "n" bytes from a descriptor. */
readn(int fd, void *vptr, size_t n)
{
size_t nleft;
ssize_t nread;
char *ptr;

ptr = vptr;
nleft = n;
while (nleft > 0) {
if ( (nread = read(fd, ptr, nleft)) < 0) {
if (errno == EINTR)
nread = 0; /* and call read() again */
else
return(-1);
} else if (nread == 0)
break; /* EOF */

nleft -= nread;
ptr += nread;
}
return(n - nleft); /* return >= 0 */
}

Henry Townsend

2006-09-11, 1:31 pm

Nils O. Selåsdal wrote:
> Henry Townsend wrote:
> This is from the UNP book, use that so you don't need to speculate how
> much read will read under certain circumstances..

Thank you all, but particularly for this code. Looks like I can plug
this one in and stop worrying.

Thanks,
HT

David Schwartz

2006-09-12, 1:25 am