|
Home > Archive > Unix Programming > December 2004 > how to tell if an open file has been removed
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
how to tell if an open file has been removed
|
|
| Ara.T.Howard 2004-12-09, 6:20 pm |
|
if i have the following logic
void
daemon ()
{
...
fs = fopen ("pidfile", "a+");
ret = non_blocking_write_lock (fs);
if (ret == 0)
{
write_pid (fs);
}
else
{
exit (42);
}
...
}
in a process (ensures only one instance of a daemon process running at once).
and a user accidentally does
rm pidfile
it will become possible for another daemon to start. in general, is there a
way to tell if a file has been removed while open?
kind regards.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| Stephane CHAZELAS 2004-12-09, 6:20 pm |
| 2004-12-9, 11:10(-07), Ara.T.Howard:
[...]
> fs = fopen ("pidfile", "a+");
[...]
> in a process (ensures only one instance of a daemon process running at once).
> and a user accidentally does
>
> rm pidfile
>
> it will become possible for another daemon to start. in general, is there a
> way to tell if a file has been removed while open?
[...]
If the number of links reaches 0, then you're sure it has been
removed. If not, it may not have been deleted, but it might have
been renamed or linked and then removed.
See fstat(2) and fileno(3) to get the number of links.
--
Stephane
| |
| Rich Teer 2004-12-09, 6:20 pm |
| On Thu, 9 Dec 2004, Ara.T.Howard wrote:
> }
> else
> {
> exit (42);
> }
Yuck! Horrible GNU indent "style"...
> it will become possible for another daemon to start. in general, is there a
> way to tell if a file has been removed while open?
You could always call stat and check its link count.
But I wonder how important this is? IF you put the PID file in a
place where it's not likely to be deleted by accident, you should
be "safe enough".
--
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich
| |
| Pascal Bourguignon 2004-12-09, 6:20 pm |
| "Ara.T.Howard" <Ara.T.Howard@noaa.gov> writes:
> in general, is there a
> way to tell if a file has been removed while open?
fstat and check the number of links.
--
__Pascal Bourguignon__ http://www.informatimago.com/
The world will now reboot; don't bother saving your artefacts.
| |
| Ara.T.Howard@noaa.gov 2004-12-09, 6:20 pm |
| On Thu, 9 Dec 2004, Rich Teer wrote:
> On Thu, 9 Dec 2004, Ara.T.Howard wrote:
>
>
> Yuck! Horrible GNU indent "style"...
;-). horrible and automatic - i always use the 'indent' program. any
formatting one has to maintain by hand is evil.
>
> You could always call stat and check its link count.
hmmm.
harp:~ > gcc a.c && a.out
st_nlink <1>
harp:~ > cat a.c
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
int
main (argc, argv)
int argc;
char **argv;
{
FILE *fs;
char *path = "barfoo";
int status;
struct stat buf;
fs = fopen (path, "a+");
if (fork)
{
wait (&status);
stat (path, &buf);
printf ("st_nlink <%u>\n", buf.st_nlink);
}
else
{
unlink (path);
fs = fopen (path, "a+");
return (0);
}
return (0);
}
nope. ;-(
i think that approach will always contain a race condition.
> But I wonder how important this is? IF you put the PID file in a place
> where it's not likely to be deleted by accident, you should be "safe
> enough".
well - that was my initial thought. however, it seems our operators have
sticky fingers and managed to do exactly this. of course they claimed not to
but i could see it in the /proc file system (dead link on the fd referencing
the pidfile).
i'm thinking there is no way to accomplish what i want to do... my current
scheme is:
processing_loop do
check_pidfile_contents or abort
processing
end
eg. i check the pidfile contents (carefully handling errors like ENOENT,
ESTALE, etc.). i think it's reasonable to assume that iff someone removed the
file, overwrote it, or modified it - it would not have the same contents.
this solution is still imperfect however...
kind regards.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| Ara.T.Howard@noaa.gov 2004-12-09, 6:20 pm |
| On Thu, 9 Dec 2004, Pascal Bourguignon wrote:
> "Ara.T.Howard" <Ara.T.Howard@noaa.gov> writes:
>
> fstat and check the number of links.
it's a race condition.
cheers.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| Ara.T.Howard@noaa.gov 2004-12-09, 6:20 pm |
| On Thu, 9 Dec 2004, Stephane CHAZELAS wrote:
> 2004-12-9, 11:10(-07), Ara.T.Howard:
> [...]
> [...]
> [...]
>
> If the number of links reaches 0, then you're sure it has been
> removed. If not, it may not have been deleted, but it might have
> been renamed or linked and then removed.
>
> See fstat(2) and fileno(3) to get the number of links.
>
> --
> Stephane
see my other replies...
cheers.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| hymie! 2004-12-09, 6:20 pm |
| In our last episode, the evil Dr. Lacto had captured our hero,
"Ara.T.Howard" <Ara.T.Howard@noaa.gov>, who said:
>in a process (ensures only one instance of a daemon process running at once).
>and a user accidentally does
>
> rm pidfile
>
>it will become possible for another daemon to start. in general, is there a
>way to tell if a file has been removed while open?
If I understand the nature of your question, you're asking if program B
can determine that program A has opened a file, formerly named "pidfile",
which has since been removed, but might still be open.
No. Your first problem is that filenames are part of the directory that
holds the file, not the file itself. Once the file has been rm'd , it
doesn't have a name anymore. (Technically, it never did.)
hymie! http://www.smart.net/~hymowitz hymie@lactose.smart.net
========================================
=======================================
| |
| Hecate-Htoniya 2004-12-09, 6:20 pm |
| Ara.T.Howard wrote:[vbcol=seagreen]
> --
> ========================================
=======================================
>
> | EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
> | PHONE :: 303.497.6469
> | When you do something, you should burn yourself completely, like a good
> | bonfire, leaving no trace of yourself. --Shunryu Suzuki
> ========================================
=======================================[
/vbcol]
correct your signature ("--" --> "-- ")
not thank me (:
--
Nukuoro Atoll, Federated States of Micronesia
#311791975
Current Music: *Ishq - Arc*
| |
| Rich Teer 2004-12-09, 6:20 pm |
| On Thu, 9 Dec 2004 Ara.T.Howard@noaa.gov wrote:
> i think that approach will always contain a race condition.
Agreed; I think it's unavoidable.
Sometimes there are no technical solutions to "people" problems...
> well - that was my initial thought. however, it seems our operators have
> sticky fingers and managed to do exactly this. of course they claimed not to
> but i could see it in the /proc file system (dead link on the fd referencing
> the pidfile).
Hmmm. The application of a clue-by-four might help here. :-)
--
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich
| |
| Ara.T.Howard@noaa.gov 2004-12-09, 6:20 pm |
| On Thu, 9 Dec 2004, Rich Teer wrote:
> On Thu, 9 Dec 2004 Ara.T.Howard@noaa.gov wrote:
>
>
> Agreed; I think it's unavoidable.
>
> Sometimes there are no technical solutions to "people" problems...
>
>
> Hmmm. The application of a clue-by-four might help here. :-)
>
> --
> Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"
>
> President,
> Rite Online Inc.
>
> Voice: +1 (250) 979-1638
> URL: http://www.rite-group.com/rich
lol.
thanks for all the suggestions!
kind regards.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| Pascal Bourguignon 2004-12-09, 6:20 pm |
| Ara.T.Howard@noaa.gov writes:
> On Thu, 9 Dec 2004, Pascal Bourguignon wrote:
>
>
> it's a race condition.
What is not on unix?
--
__Pascal Bourguignon__ http://www.informatimago.com/
The world will now reboot; don't bother saving your artefacts.
| |
| Mark Rafn 2004-12-10, 2:53 am |
| Ara.T.Howard <Ara.T.Howard@noaa.gov> wrote:
....
>in a process (ensures only one instance of a daemon process running at once).
>and a user accidentally does
> rm pidfile
>it will become possible for another daemon to start.
I see there's been some discussion about this already. Generally, don't put
the file in a place where it is likely to be deleted, and if it's worth fixing
this, you have a few basic choices:
1) don't use the filesystem for this kind of locking. There is no way to
fix the race completely, though you can minimize it. Use some other
IPC mechanism for processes to determine if they're alone if you need
100% completeness.
2) reduce the harm of having 2 processes running. If the program routinely
checks it's lockfile to make sure it still has the right pid in it, and
exits if it's been replaced (and replaces it if it's removed), you
won't have 2 running for very long.
3) start the process at boot, and leave it running forever. If you never
need to start a new one, you never need to check if one's already
running.
>in general, is there a way to tell if a file has been removed while open?
If stat(filename) gives you a different dev or inode than fstat(filehandle) on
your open file, then it's been removed and replaced. Since you're putting
your pid in the file, you can also check the contents and see if it's still
the same.
--
Mark Rafn dagon@dagon.net <http://www.dagon.net/>
| |
| Stephane CHAZELAS 2004-12-10, 7:54 am |
| 2004-12-9, 13:51(-07), Ara.T.Howard@noaa.gov:
> On Thu, 9 Dec 2004, Pascal Bourguignon wrote:
>
>
> it's a race condition.
[...]
Which race condition? In your other posts, you've used stat(2),
we were talking of fstat(2) (on the already opened file).
If the link count is 0, there's no way it can be increased back
$ zmodload zsh/stat
$ exec 3> foo
$ stat +nlink foo # stat(2)
1
$ stat +nlink -f 3 # fstat(2)
1
$ rm foo
$ stat +nlink foo # stat(2)
stat: foo: no such file or directory
(1)$ stat +nlink -f 3
0
$ echo newfoo > foo
$ stat +nlink foo # stat(2)
1
$ stat +nlink -f 3 # fstat(2)
0
$ rm foo
$ ln /proc/self/fd/3 foo
ln: creating hard link `foo' to `/proc/self/fd/3': Invalid cross-device link
nlink == 0 on an opened file means that the file has been
removed and that it can't be linked back to the file system (of
course, a new file by the same name can be created, but that's
another issue).
--
Stephane
| |
| Ara.T.Howard@noaa.gov 2004-12-10, 5:55 pm |
| On Thu, 9 Dec 2004, Mark Rafn wrote:
> I see there's been some discussion about this already. Generally, don't put
> the file in a place where it is likely to be deleted, and if it's worth fixing
> this, you have a few basic choices:
> 1) don't use the filesystem for this kind of locking. There is no way to
> fix the race completely, though you can minimize it. Use some other
> IPC mechanism for processes to determine if they're alone if you need
> 100% completeness.
good idea.
> 2) reduce the harm of having 2 processes running. If the program routinely
> checks it's lockfile to make sure it still has the right pid in it, and
> exits if it's been replaced (and replaces it if it's removed), you
> won't have 2 running for very long.
this is done - all access to shared storage is protected by transactions - in
fact there is zero harm of having two running - it's just a bugette.
> 3) start the process at boot, and leave it running forever. If you never
> need to start a new one, you never need to check if one's already
> running.
the daemon is an 'immortal daemon'. a crontab entry runs every 15 minutes and
starts it if it is not already running.
>
> If stat(filename) gives you a different dev or inode than fstat(filehandle) on
> your open file, then it's been removed and replaced. Since you're putting
> your pid in the file, you can also check the contents and see if it's still
> the same.
fstat IS the secret. however the dev an ino fields aren't - at least on my
system:
harp:~ > gcc ./a.c -o a && a
[22713] - st_dev <774>
[22713] - st_ino <32892>
[22713] - st_nlink <1>
[22714] - recreating <barfoo>
[22713] - st_dev <774>
[22713] - st_ino <32892>
[22713] - st_nlink <0>
harp:~ > cat a.c
#include <stdlib.h>
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
int
main (argc, argv)
int argc;
char **argv;
{
FILE *fs;
char *path = "barfoo";
int status;
struct stat buf;
pid_t cid;
fs = fopen (path, "a+");
fstat (fileno (fs), &buf);
fprintf (stderr, "[%u] - st_dev <%u>\n", getpid (), buf.st_dev);
fprintf (stderr, "[%u] - st_ino <%u>\n", getpid (), buf.st_ino);
fprintf (stderr, "[%u] - st_nlink <%u>\n", getpid (), buf.st_nlink);
if (fork ())
{
wait (&status);
fstat (fileno (fs), &buf);
fprintf (stderr, "[%u] - st_dev <%u>\n", getpid (), buf.st_dev);
fprintf (stderr, "[%u] - st_ino <%u>\n", getpid (), buf.st_ino);
fprintf (stderr, "[%u] - st_nlink <%u>\n", getpid (), buf.st_nlink);
return 0;
}
else
{
fprintf (stderr, "[%u] - recreating <%s>\n", getpid (), path);
unlink (path);
fs = fopen (path, "a+");
return 0;
}
}
thanks for the tip. does any one know if this is portable or why the ino and
dev fields remain the same?
kind regards.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| Ara.T.Howard@noaa.gov 2004-12-10, 5:55 pm |
| On Fri, 10 Dec 2004, Stephane CHAZELAS wrote:
> Which race condition? In your other posts, you've used stat(2), we were
> talking of fstat(2) (on the already opened file).
yes - my mistake. no race with fstat. i missed that earlier.
> If the link count is 0, there's no way it can be increased back
>
> $ zmodload zsh/stat
> $ exec 3> foo
> $ stat +nlink foo # stat(2)
> 1
> $ stat +nlink -f 3 # fstat(2)
> 1
> $ rm foo
> $ stat +nlink foo # stat(2)
> stat: foo: no such file or directory
> (1)$ stat +nlink -f 3
> 0
> $ echo newfoo > foo
> $ stat +nlink foo # stat(2)
> 1
> $ stat +nlink -f 3 # fstat(2)
> 0
> $ rm foo
> $ ln /proc/self/fd/3 foo
> ln: creating hard link `foo' to `/proc/self/fd/3': Invalid cross-device link
>
> nlink == 0 on an opened file means that the file has been
> removed and that it can't be linked back to the file system (of
> course, a new file by the same name can be created, but that's
> another issue).
see my other posts - this DOES seem to be the way to accomplish my goal.
thanks for the info!
kind regards.
-a
--
========================================
=======================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
========================================
=======================================
| |
| Michael B Allen 2004-12-16, 7:44 pm |
| On Thu, 09 Dec 2004 13:10:03 -0500, Ara.T.Howard wrote:
> if i have the following logic
>
> void
> daemon ()
> {
> ...
> }
>
> in a process (ensures only one instance of a daemon process running at
> once). and a user accidentally does
>
> rm pidfile
>
> it will become possible for another daemon to start. in general, is
> there a way to tell if a file has been removed while open?
If I may suggest a slightly different method -- I think if you use
lockf(3) on the pidfile it will still provided the desired exclusion
regardless of wheather or not the file exists in the filesystem.
See:
http://www.ioplex.com/~miallen/libmba/dl/src/daemon.c
http://www.enderunix.org/docs/eng/daemon.php
Mike
| |
|
|
Since you don't seem interested in the pid, there
is no need to store it. (otherwise the pid could be
used to kill(pid, 0), and check the errno, but this
would also cause heisen-races ;-)
If your only goal is to periodically check the
_availability_ of the _service_ (offered by the daemon)
you could use a socket , which responds to "AYT" packets.
This way you could distinguish a hanging (looping) server
from a dead one.
If you can bind() the address (say {localhost,911}) then
you are the first instance. proceed.
If you cannot bind, there is already a server process.
If it responds to your AYT - message (in time, properly),
you exit.
If it does not respond, you kill it off (need the PID here ...)
and proceed.(there wil be a problem with bind/TIME_WAIT here,
but that is better than a dead server, IMO)
Users are always a problem. They could bind your socket,
remove your pidfile, disconnect your power, etc. deal with them...
BTW periodically rewriting the pidfile could also serve as
a sign-of-life, presuming the daemon is the sole writer.
NB Maybe unix-domain sockets/named pipes could come handy here ?
HTH,
AvK
| |
| Randy Howard 2004-12-16, 7:45 pm |
| In article <Pine.LNX.4.60.0412091344190.21479@harp.ngdc.noaa.gov>,
Ara.T.Howard@noaa.gov says...
> On Thu, 9 Dec 2004, Rich Teer wrote:
>
>
> ;-). horrible and automatic - i always use the 'indent' program. any
> formatting one has to maintain by hand is evil.
You do realize that indent supports command line parameters and
configuration files so that you can change the default behavior to
one that makes more sense?
| |
| Heny Townsend 2004-12-16, 7:45 pm |
| Randy Howard wrote:
>
>
> You do realize that indent supports command line parameters and
> configuration files so that you can change the default behavior to
> one that makes more sense?
He's right. And worse, I believe different indent programs have
different default styles. I.e. only the GNU indent produces the GNU
style by default. So having a ~/.indent.pro file is essential to
producing a predictable style.
I'd show my .indent.pro but that would probably start a separate thread
on indent stylings which we could all do without.
--
Henry Townsend
|
|
|
|
|