Red Hat Topics - Determine if a file is "open"

Author

Determine if a file is "open"

Grant Schoep

2005-10-06, 5:53 pm

Not sure where to start or post this question. So just taking a few random
wacks at this.

Is there a systematic way to determine if a file is currently "open" by a
live process somewhere? I don't really care where or by what. Just that
some process actually has that file open.

Problem. We have this logging directory that a number of different apps on
this system share. They are all bad processes that like to leave these
logging files around. The data is important, I want to archive it off at
certain points. But I don't want to "move" away those files in the
directory if they are still open. There is no way by looking at the data in
the file itself to know when it is closed.

I just want to be able to look at a filename and somehow determine if some
process has it open. Note, this could be on another machine. As this is an
NFS mount point.

It doesn't matter if the solution is via C, some scripting languange, shell
language, or whatever. I just am trying to find out if it is even possible
to determine if a file is open or not.

Since it probably matters, in this case all our machines that would have
one of these files open are running RedHat ES 3. All are NFS mounting that
partition which is using the ext3 filesystem.

Scott Lurndal

2005-10-06, 8:46 pm

Grant Schoep <matobinder@idcomm.abcremooveabc.com> writes:
>Not sure where to start or post this question. So just taking a few random
>wacks at this.
>
>Is there a systematic way to determine if a file is currently "open" by a
>live process somewhere? I don't really care where or by what. Just that
>some process actually has that file open.

man lsof

>
>Problem. We have this logging directory that a number of different apps on
>this system share. They are all bad processes that like to leave these
>logging files around. The data is important, I want to archive it off at
>certain points. But I don't want to "move" away those files in the
>directory if they are still open. There is no way by looking at the data in
>the file itself to know when it is closed.

Not to worry.

A quick unix filesystem lesson:

The content of a file is represented as a set of one or more disk
blocks. Disk blocks are one or more disk sectors. Typically,
the ext2 filesystem uses 1024 byte blocks on media with 512-byte
sectors.

A file itself is represented in the filesystem by an inode. The inode
contains the metadata (out-of-band state information) for the file,
such as the last access time, the size in bytes of the file, the
access permissions for the file, and most importantly, the address(es)
of the data block(s) that contain the file data.

A directory entry contains a tuple consisting of a text name and an
inode number. Multiple directory entries in one or more directories
may refer to the same inode with different filenames (this is called a
hard link).

When you use rm(1) or unlink(2) to remove a file, it simply removes the
name from the directory entry (thus breaking the association between
the name and the inode). However, the inode will not be removed unless
its link count (i.e. the number of directory entries that refer to the
inode) is zero _AND_ the in-use count (the number of processes that have
the inode open) is zero.

This means that if a process has the file open, it will not be removed
until the process closes the file. If you move them to a directory in
the same filesystem (which is a hardlink operation followed by an unlink
operation), the process which has the file open will continue to write
to it until it closes the file descriptor. If you move the file to a
different filesystem, the mv(1) command will treat it as a cp(1) followed
by a rm(1). In the latter case, only the portion of the file that had
been written at the time of the mv(1) will be copied, and the process will
continue to write to the original inode until it closes the file, after
which the content will be lost (assuming the link count and in-use counts
have both reached zero).

>
>I just want to be able to look at a filename and somehow determine if some
>process has it open. Note, this could be on another machine. As this is an
>NFS mount point.

lsof(1) will show all open files on the system and which process has them
open.

Just beware that there is no reliable way to transform an inode number
into a filename.

scott

Grant Schoep

2005-10-12, 5:59 pm

Thanks. Its all good now. I used a combo of the output from lsof, and
direct Ruby, manip of directory entries. Ruby rocks, this was my first Ruby
tool. Its another scripting language like Perl, but is more directly object
oriented, and, well, its pretty readable and reusable too. We now can
accuratly type "ls *.log" without it saying argument list too long. This
rules.I actually wrote two scripts,one in c-shell, and one in Ruby. On our
one system with +30,000 plus files, it took Ruby about 10 seconds to sort
those files out in a year_month directory structure. I restored it, and
with csh, it took almost 7 minutes.