|
Home > Archive > Unix Programming > April 2005 > S_ISREG/File Race Conditions
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
S_ISREG/File Race Conditions
|
|
| Josh Birnbaum 2005-04-26, 2:49 am |
| Hello All,
I have a question regarding the use of the S_ISREG file type macro.
Firstly, some code:
FILE *file = NULL;
struct stat statBuf;
if( ( file = fopen( "/my/file", "r" ) ) == NULL )
{
perror( "fopen" );
return (-1);
}
if( ( fstat( fileno( file ), &statBuf ) ) != 0 )
{
perror( "fstat" );
return (-1);
}
if( ! ( S_ISREG( statBuf.st_mode ) ) )
{
fprintf( stderr, "ERROR: /my/file is not a regular file\n" );
return (-1);
}
In checking the file type of /my/file, and knowing that the file should
always be of type "regular", is it enough for me to use the S_ISREG file
type macro to determine if /my/regular/file is actually a regular file?
Or, do I also need to, say, lstat(2) it to make sure it isn't a link or
do anything else?
One more question. As I understand it, stat(2)ing a file and then
calling
fopen(3S) can result in a race condition if the file is replaced in bet-
ween the two calls.
As a remedy to this problem, I have read (and would like to confirm)
that calling fopen(3S) on the file and then fstat(2)ing it eliminates
the threat
of the file race condition.
Any feedback would be appreciated.
Thanks,
Josh.
| |
| junky_fellow@yahoo.co.in 2005-04-26, 2:49 am |
|
Josh Birnbaum wrote:
> Hello All,
>
> I have a question regarding the use of the S_ISREG file type macro.
> Firstly, some code:
>
> FILE *file = NULL;
> struct stat statBuf;
>
> if( ( file = fopen( "/my/file", "r" ) ) == NULL )
> {
> perror( "fopen" );
> return (-1);
> }
>
> if( ( fstat( fileno( file ), &statBuf ) ) != 0 )
> {
> perror( "fstat" );
> return (-1);
> }
>
> if( ! ( S_ISREG( statBuf.st_mode ) ) )
> {
> fprintf( stderr, "ERROR: /my/file is not a regular file\n" );
> return (-1);
> }
>
> In checking the file type of /my/file, and knowing that the file
should
> always be of type "regular", is it enough for me to use the S_ISREG
file
> type macro to determine if /my/regular/file is actually a regular
file?
> Or, do I also need to, say, lstat(2) it to make sure it isn't a link
or
> do anything else?
>
> One more question. As I understand it, stat(2)ing a file and then
> calling
> fopen(3S) can result in a race condition if the file is replaced in
bet-
> ween the two calls.
> As a remedy to this problem, I have read (and would like to confirm)
> that calling fopen(3S) on the file and then fstat(2)ing it eliminates
> the threat
> of the file race condition.
> Any feedback would be appreciated.
>
> Thanks,
>
> Josh.
If the file is a link, you should use lstat.
Successful stat() of a file does not guarantee that next open of the
file would be successful. A file may be deleted in between stat and
open.
If you open a file before stat, and some other process deletes the
file,
in that case only the directory entry of the file is removed.
The process that has opened the file can read/write or fstat the file
using the "file descriptor" that was returned during open.
If you stat the removed file by the "complete file name" it would
FAIL.
All other open of the removed file would fail.
After the last close of file, the data/metadata of the file be
removed and all further operation on the file will fail.
| |
| Casper H.S. Dik 2005-04-26, 2:49 am |
| Josh Birnbaum <engineer@noorg.org> writes:
>Hello All,
>I have a question regarding the use of the S_ISREG file type macro.
>Firstly, some code:
>FILE *file = NULL;
>struct stat statBuf;
>if( ( file = fopen( "/my/file", "r" ) ) == NULL )
>{
> perror( "fopen" );
> return (-1);
> }
>if( ( fstat( fileno( file ), &statBuf ) ) != 0 )
>{
> perror( "fstat" );
> return (-1);
> }
>if( ! ( S_ISREG( statBuf.st_mode ) ) )
>{
> fprintf( stderr, "ERROR: /my/file is not a regular file\n" );
> return (-1);
> }
>In checking the file type of /my/file, and knowing that the file should
>always be of type "regular", is it enough for me to use the S_ISREG file
>type macro to determine if /my/regular/file is actually a regular file?
>Or, do I also need to, say, lstat(2) it to make sure it isn't a link or
>do anything else?
Yes, it is sufficient; once you open the file there's a fixed correspondence
between the fd and the file object; it cannot change type so doing fopen
and fstat will return information about the file you opened, but possibly
no longer the path you opened it with.
>One more question. As I understand it, stat(2)ing a file and then
>calling
>fopen(3S) can result in a race condition if the file is replaced in bet-
>ween the two calls.
Correct.
>As a remedy to this problem, I have read (and would like to confirm)
>that calling fopen(3S) on the file and then fstat(2)ing it eliminates
>the threat
>of the file race condition.
>Any feedback would be appreciated.
yes, unless you don't want to follow symlinks.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Josh Birnbaum 2005-04-26, 5:59 pm |
| Hi junky_fellow,
> If the file is a link, you should use lstat.
No, it should _never_ be a link. All I'm trying to establish is
that if I *know* that the file should always be of type "regular",
is it enough for me to use the S_ISREG macro to determine if it
is, or is not, a regular file? Here's the call sequence:
if( ( file = fopen( "/my/file", "r" ) ) == NULL )
{
perror( "fopen" );
return (-1);
}
if( ( fstat( fileno( file ), &statBuf ) ) != 0 )
{
perror( "fstat" );
return (-1);
}
if( ! ( S_ISREG( statBuf.st_mode ) ) )
{
fprintf( stderr, "ERROR: /my/file is not a regular file\n" );
return (-1);
}
If S_ISREG does not fail, can I be reasonably sure that /my/file is
of type "regular"?
> Successful stat() of a file does not guarantee that next open of the
> file would be successful. A file may be deleted in between stat and
> open.
I only fopen once, throughout this whole operation. And, I'm not stating
a file I didn't yet call fopen on. I'm fstating a file that was just
opened via fopen, to avoid a race condition.
> If you open a file before stat, and some other process deletes the
> file,
> in that case only the directory entry of the file is removed.
> The process that has opened the file can read/write or fstat the file
> using the "file descriptor" that was returned during open.
OK.
> If you stat the removed file by the "complete file name" it would
> FAIL.
Because the directory entry has been removed/unlinked.
> All other open of the removed file would fail.
Makes sense.
> After the last close of file, the data/metadata of the file be
> removed and all further operation on the file will fail.
Again, makes sense. Because the link count drops to 0?
What do you think of my comments above?
Thanks Again,
Josh.
| |
| Josh Birnbaum 2005-04-26, 5:59 pm |
| Hi Casper,
> Josh Birnbaum <engineer@noorg.org> writes:
>
>
>
>
>
>
>
>
> Yes, it is sufficient; once you open the file there's a fixed correspondence
> between the fd and the file object; it cannot change type so doing fopen
> and fstat will return information about the file you opened
OK. By extention, I take this to mean that S_ISREG will be reliable, at
that
point in time.
> but possibly no longer the path you opened it with.
Because the file could be moved or deleted, in the interrim?
>
> Correct.
>
>
> yes, unless you don't want to follow symlinks.
The file in question should never be a link. If it is a link, my
previous call
to the S_ISREG macro would fail, as per your explanation, above.
Correct?
If S_ISREG fails, my fprintf error output kicks in and I return -1.
Thanks Casper.
Yours,
Josh.
> Casper
> --
> Expressed in this posting are my opinions. They are in no way related
> to opinions held by my employer, Sun Microsystems.
> Statements on Sun products included here are not gospel and may
> be fiction rather than truth.
| |
| Eric Sosman 2005-04-26, 5:59 pm |
|
Josh Birnbaum wrote:
> [ fopen() / fstat() / S_ISREG() ...]
>
> The file in question should never be a link. If it is a link, my
> previous call
> to the S_ISREG macro would fail, as per your explanation, above.
> Correct?
Not quite. The file will never be a link because the
fopen() will already have followed any links to their
targets. The fstat() then operates on the ultimate target
of the chain of links -- this might be a regular file or a
directory or a device special file or the Creature from the
Black Lagoon, but one thing it most definitely will *not*
be is a symbolic link.
--
Eric.Sosman@sun.com
| |
| Casper H.S. Dik 2005-04-26, 5:59 pm |
| Josh Birnbaum <engineer@noorg.org> writes:
>if( ! ( S_ISREG( statBuf.st_mode ) ) )
>{
> fprintf( stderr, "ERROR: /my/file is not a regular file\n" );
> return (-1);
> }
>If S_ISREG does not fail, can I be reasonably sure that /my/file is
>of type "regular"?
No; you can only assume that /my/file points to a regular file;
one typical method is:
open
fstat
lstat
then compare the results of step 2 and 3.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Gordon Burditt 2005-04-26, 5:59 pm |
| >> If the file is a link, you should use lstat.
>
>No, it should _never_ be a link. All I'm trying to establish is
>that if I *know* that the file should always be of type "regular",
>is it enough for me to use the S_ISREG macro to determine if it
>is, or is not, a regular file?
If you *know* that the file should always be of type "regular",
why check at all? Or why not just check for existence (by
trying to open it for read with fopen())?
>If S_ISREG does not fail, can I be reasonably sure that /my/file is
>of type "regular"?
>
>
>I only fopen once, throughout this whole operation. And, I'm not stating
>a file I didn't yet call fopen on. I'm fstating a file that was just
>opened via fopen, to avoid a race condition.
What are you trying to accomplish?
If you want "is the file that I opened as /my/file a regular file"?, you're
safe, and further operations using that file descriptor also refer to the
same file (but it may not have the same, or any, name as before).
If you want "is the file NOW named /my/file a regular file"?, you're NOT
safe; you have a race condition. Try using stat(), which checks the
type and the name at the same time, but if you intend doing anything further
with the same file, you *still* have a race condition.
>
>OK.
>
>
>Because the directory entry has been removed/unlinked.
>
Unless someone creates another one under that name first.[vbcol=seagreen]
>
>Makes sense.
Gordon L. Burditt
| |
| Josh Birnbaum 2005-04-26, 5:59 pm |
| Hi Gordon,
>
> What are you trying to accomplish?
Having just fopened, fstated and then S_ISREG'd the file, I want to read
select lines from it via fgets and then pass those lines to sscanf for
ultimate output via printf. That's the actual sequence. I never let
go of the fopened returned descriptor, throughout all of this.
> If you want "is the file that I opened as /my/file a regular file"?, you're
> safe, and further operations using that file descriptor also refer to the
> same file (but it may not have the same, or any, name as before).
Right. The operative words here, at least in my mind, are "and further
operations using that file descriptor also refer to the same file"
The difference in name or the lack of a name could be because
someone/thing
changes the file name in mid-stream and, as a result, it's directory
entry?
> If you want "is the file NOW named /my/file a regular file"?, you're NOT
> safe; you have a race condition.
"NOW", as in having closed the file (and released its descriptor), and
then
opening the file again, at some later time?
> Try using stat(), which checks the type and the name at the same time,
> but if you intend doing anything further with the same file, you *still*
> have a race condition.
I was calling stat() before fopening the file. I've changed this to
firstly
calling fopen and then fstating the returned descriptor, in the hopes of
avoiding the race condition.
So, even if I fopen, get the descriptor, do my fstat, and pass the
descriptor
to fgets, I'm still in trouble?
>
> Unless someone creates another one under that name first.
>
> Gordon L. Burditt
Thanks,
Josh.
| |
| Josh Birnbaum 2005-04-26, 5:59 pm |
| Hi Casper,
>
>
> No; you can only assume that /my/file points to a regular file;
> one typical method is:
>
> open
> fstat
> lstat
>
> then compare the results of step 2 and 3.
Ok. In searching through the comp.unix.programmer archive, I found
something dated 1998 from you:
"then you can lstat/fstat at your leisure and check link counts,
st_mode, st_ino and st_dev"
So, one should compare these stat struct members, as returned by
fstat and lstat, for a more complete check, it would seem.
(A passage from the "Lab engineers check list for writing secure
Unix code", adapted from Practical UNIX and Internet Security,
says roughly the same thing, though that says to lstat then open
then fstat and then compare the lstat and fstat struct members).
> Casper
> --
> Expressed in this posting are my opinions. They are in no way related
> to opinions held by my employer, Sun Microsystems.
> Statements on Sun products included here are not gospel and may
> be fiction rather than truth.
Thanks,
Josh.
| |
| Gordon Burditt 2005-04-26, 5:59 pm |
| >> >I only fopen once, throughout this whole operation. And, I'm not stating
>
>Having just fopened, fstated and then S_ISREG'd the file, I want to read
>select lines from it via fgets and then pass those lines to sscanf for
>ultimate output via printf. That's the actual sequence. I never let
>go of the fopened returned descriptor, throughout all of this.
Once you get the descriptor, operations done through that descriptor
refer to the same file (*NOT* necessarily the same name). This is
the key to what you're trying to do.
>
>Right. The operative words here, at least in my mind, are "and further
>operations using that file descriptor also refer to the same file"
>The difference in name or the lack of a name could be because
>someone/thing
>changes the file name in mid-stream and, as a result, it's directory
>entry?
If someone goes nuts renaming stuff, you could end up with a directory, or
a different file, or a named nuclear weapon (see: Deathstation 9000) or
whatever. Hang on to the file descriptor and USE it, and you'll keep
the reference to the same file. Some people, however, want to keep
the reference to the same *NAME*, and that's not going to work.
>
>"NOW", as in having closed the file (and released its descriptor), and
>then
>opening the file again, at some later time?
Yes, or using stat() (not fstat()), or link(), or unlink() or
rename() or anything else that refers to the file independently by
name, whether or not you've closed the original file descriptor
(there is no funlink() or fremove(), for example). The key here
is that stat(), link(), unlink(), and rename() don't take a file
descriptor, and 3 of the 4 don't have variants that do.
For some purposes, it's the *NAME*, not the file, that's important
(lock files being one example). Renaming the lock file with flock()
held on it is a problem because other processes going after the
same *NAME* will refer to a different file which won't have the
flock() on it.
>
>I was calling stat() before fopening the file. I've changed this to
>firstly
>calling fopen and then fstating the returned descriptor, in the hopes of
>avoiding the race condition.
Assuming you want the same file you first did the fopen() on, that's
good.
>So, even if I fopen, get the descriptor, do my fstat, and pass the
>descriptor
>to fgets, I'm still in trouble?
No, you're fine. But note that there isn't a funlink(), fremove(),
flink(), or frename(). Sometimes you gotta use the name for some
things. And each time you do, you won't necessarily get the same
file as the last reference did.
>
>Thanks,
>
>Josh.
Gordon L. Burditt
| |
| Ian Pilcher 2005-04-26, 5:59 pm |
| Eric Sosman wrote:
> Not quite. The file will never be a link because the
> fopen() will already have followed any links to their
> targets. The fstat() then operates on the ultimate target
> of the chain of links -- this might be a regular file or a
> directory or a device special file or the Creature from the
> Black Lagoon, but one thing it most definitely will *not*
> be is a symbolic link.
I think that the OP wants to protect against using a pathname that
refers to a symbolic link. On Linux, he could do this with
open(..., O_NOFOLLOW), fstat(), and fdopen(). The open(2) man page on
Linux indicates that O_NOFOLLOW is an extension, however; I can't think
of a non-racy way to do this in its absence.
--
========================================
================================
Ian Pilcher i.pilcher@comcast.net
========================================
================================
| |
| Casper H.S. Dik 2005-04-27, 7:56 am |
| Josh Birnbaum <engineer@noorg.org> writes:
>Hi Casper,
[vbcol=seagreen]
>Ok. In searching through the comp.unix.programmer archive, I found
>something dated 1998 from you:
>"then you can lstat/fstat at your leisure and check link counts,
>st_mode, st_ino and st_dev"
>So, one should compare these stat struct members, as returned by
>fstat and lstat, for a more complete check, it would seem.
>(A passage from the "Lab engineers check list for writing secure
> Unix code", adapted from Practical UNIX and Internet Security,
> says roughly the same thing, though that says to lstat then open
> then fstat and then compare the lstat and fstat struct members).
Lstat before or after the open?
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Casper H.S. Dik 2005-04-27, 7:56 am |
| Ian Pilcher <i.pilcher@comcast.net> writes:
>Eric Sosman wrote:
[vbcol=seagreen]
>I think that the OP wants to protect against using a pathname that
>refers to a symbolic link. On Linux, he could do this with
>open(..., O_NOFOLLOW), fstat(), and fdopen(). The open(2) man page on
>Linux indicates that O_NOFOLLOW is an extension, however; I can't think
>of a non-racy way to do this in its absence.
Quite; Solaris 10 also has O_NOFOLLOW for this purpose.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Josh Birnbaum 2005-04-27, 5:58 pm |
| "Casper H.S. Dik" wrote:
>
> Josh Birnbaum <engineer@noorg.org> writes:
Hi Casper,
>
>
>
>
> Lstat before or after the open?
Before.
Here's the portion of the check list that discusses this:
- lstat() the path, check that lstat succeeded
- check that it's acceptable (eg, not a symlink)
- open() (without O_CREAT), check that the open succeeded /*I'm using
fopen(3) */
- fstat() the fd returned by open
- if the lstat and fstat st_ino and st_dev fields match,
accept.
So, it's lstat() then open() then fstat().
Thanks,
Josh.
> Casper
> --
> Expressed in this posting are my opinions. They are in no way related
> to opinions held by my employer, Sun Microsystems.
> Statements on Sun products included here are not gospel and may
> be fiction rather than truth.
| |
| Ian Pilcher 2005-04-27, 5:58 pm |
| Josh Birnbaum wrote:
> So, it's lstat() then open() then fstat().
Note that not all filesystems actually guarantee unique inodes. AFS,
for example, can actually have more than 2^32 files. In the absence of
O_NOFOLLOW, however, it's the best you can do.
--
========================================
================================
Ian Pilcher i.pilcher@comcast.net
========================================
================================
| |
| Josh Birnbaum 2005-04-28, 2:49 am |
| Hi Ian,
> Josh Birnbaum wrote:
>
> Note that not all filesystems actually guarantee unique inodes. AFS,
> for example, can actually have more than 2^32 files.
So the inode numbers wrap?
> In the absence of O_NOFOLLOW, however, it's the best you can do.
This is what I've done:
char *filePath = NULL;
struct stat lstatBuf;
struct stat fstatBuf;
FILE *file = NULL;
/* filePath = "./myregularfile.link"; */
filePath = "./myregularfile";
if( ( lstat( filePath, &lstatBuf ) ) != 0 )
{
perror( "lstat" );
return (-1);
}
if( S_ISLNK( lstatBuf.st_mode ) )
{
fprintf( stderr, "ERROR: ./myregularfile is a symbolic link\n" );
return (-1);
}
if( ( file = fopen( "./myregularfile", "r" ) ) == NULL )
{
perror( "fopen" );
return (-1);
}
if( ( fstat( fileno( file ), &fstatBuf ) ) != 0 )
{
perror( "fstat" );
return (-1);
}
if( ( ( lstatBuf.st_ino ) != ( fstatBuf.st_ino ) ) ||
( ( lstatBuf.st_dev ) != ( fstatBuf.st_dev ) ) )
{
fprintf( stderr, "ERROR: ./myregularfile attribute inconsistency\n"
);
return (-1);
}
I've experimented via creating a link to ./myregularfile
(./myregularfile.link), assigning that link path to filePath, and then
running
it through gdb(1). The S_ISLINK macro catches that ./myregularfile is a
link
and the program exits -1. If I comment out the S_ISLINK check, recompile
and
run the program, the st_ino/st_dev code catches the inconsistency and
the
program exits -1.
If the file is a regular file and the st_ino/st_dev code sees the same
values for these struct members in both lstatBuf and fstatBuf, all is
ok and I exit 0;
I think that this is the way I'm going to go.
Any comments are welcome.
Yours,
Josh.
> ========================================
================================
> Ian Pilcher i.pilcher@comcast.net
> ========================================
================================
|
|
|
|
|