|
Home > Archive > Unix Programming > July 2007 > Files in a directory oldest to newest
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Files in a directory oldest to newest
|
|
| jnbbender@gmail.com 2007-07-23, 7:20 pm |
| I need to write a small utility to purge files from a directory once
the directory has reached a certain threshold. I'm wondering if there
is a function of quick way for me to do this. Currently I am reading
in every file from the directory doing an lstat for the files mtime,
sorting them from there, then deleting them once I've reached my safe
capacity, i.e. the directory is now only 90% full.
Any help would be great.
Thanks
| |
| Eric Sosman 2007-07-23, 7:20 pm |
| jnbbender@gmail.com wrote On 07/23/07 14:55,:
> I need to write a small utility to purge files from a directory once
> the directory has reached a certain threshold. I'm wondering if there
> is a function of quick way for me to do this. Currently I am reading
> in every file from the directory doing an lstat for the files mtime,
> sorting them from there, then deleting them once I've reached my safe
> capacity, i.e. the directory is now only 90% full.
My first thought was "man find", but I'm not sure
just what your "threshold" is or how you decide that a
directory is "90% full." If you'd give a little more
detail on these points, maybe someone will have a better
suggestion.
--
Eric.Sosman@sun.com
| |
| Ben Bacarisse 2007-07-23, 7:20 pm |
| jnbbender@gmail.com writes:
> I need to write a small utility to purge files from a directory once
> the directory has reached a certain threshold. I'm wondering if there
> is a function of quick way for me to do this. Currently I am reading
> in every file from the directory doing an lstat for the files mtime,
> sorting them from there, then deleting them once I've reached my safe
> capacity, i.e. the directory is now only 90% full.
>
> Any help would be great.
If you are using relatively recent Linux, there is the inotify
interface (man 7 inotify) and I image that other systems have
something similar (but I don't know they do!).
--
Ben.
| |
| jnbbender@gmail.com 2007-07-23, 7:20 pm |
| On Jul 23, 3:37 pm, Ben Bacarisse <ben.use...@bsb.me.uk> wrote:
> jnbben...@gmail.com writes:
>
>
> If you are using relatively recent Linux, there is the inotify
> interface (man 7 inotify) and I image that other systems have
> something similar (but I don't know they do!).
>
> --
> Ben.
I am trying to do this in software so command line stuff/scripts ,etc
are out. I am monitoring the capacity of a directory and once it
reaches a certain threshold (say 98%) I need to start dumping files
until I reach a "safe" capacity of 90%.
I am also not on a Linux box, I'm on a Solaris system.
| |
| Ben Bacarisse 2007-07-23, 7:20 pm |
| jnbbender@gmail.com writes:
> On Jul 23, 3:37 pm, Ben Bacarisse <ben.use...@bsb.me.uk> wrote:
>
> I am trying to do this in software so command line stuff/scripts ,etc
> are out.
inotify is a kernel interface. All it would have done is save you
having to repeatedly scan the directory -- you get to do a blocking
read on a file descriptor which, when it returns, tells you about
changes to the files/directories you asked to be notified of.
> I am monitoring the capacity of a directory and once it
> reaches a certain threshold (say 98%) I need to start dumping files
> until I reach a "safe" capacity of 90%.
>
> I am also not on a Linux box, I'm on a Solaris system.
Ah. There may be something similar, but I don't know. Monitoring
files system changes without repeated polling is a common
requirement.
If, however, your real task is to prevent a file system from filling
up, there may be other system interfaces that are even more
appropriate.
--
Ben.
| |
| Logan Shaw 2007-07-24, 1:22 am |
| jnbbender@gmail.com wrote:
> On Jul 23, 3:37 pm, Ben Bacarisse <ben.use...@bsb.me.uk> wrote:
[vbcol=seagreen]
[vbcol=seagreen]
> I am trying to do this in software so command line stuff/scripts ,etc
> are out.
You have a strange definition of "software". :-)
> I am monitoring the capacity of a directory and once it
> reaches a certain threshold (say 98%) I need to start dumping files
> until I reach a "safe" capacity of 90%.
As far as I know, the way you're doing it is the most reasonable
way that's also portable.
Something like inotify could save you from having to poll periodically
and scan the entire directory when you want to purge, but I'm not
sure this is really a performance advantage. In particular, if the
files in the directory change a lot, you will probably process so
many file change notification events that you will do more work
than just scanning periodically.
If you have control over the creation of the files, you could put
something in the filename that lets you determine which is newest
(i.e. which one to throw out first). For example, you could put an
increasing integer somewhere in the filename. This would save you
from having to do an lstat() on every file at least. Then you'd
only have to read the directory itself. That could save a lot of
I/O. On the other hand, if you're asking the filesystem how much
space is left after every deletion, that might overwhelm any
performance improvements you can eek out somewhere else...
- Logan
| |
| rmurtagh@optonline.net 2007-07-30, 7:19 pm |
| On Jul 23, 2:55 pm, jnbben...@gmail.com wrote:
> I need to write a small utility to purge files from a directory once
> the directory has reached a certain threshold. I'm wondering if there
> is a function of quick way for me to do this. Currently I am reading
> in every file from the directory doing an lstat for the files mtime,
> sorting them from there, then deleting them once I've reached my safe
> capacity, i.e. the directory is now only 90% full.
>
> Any help would be great.
>
> Thanks
The following script will check a location to see if a 98% threshold
has been exceeded, but it is just already doing what you are doing in
a different manner. If it has been exceeded the oldest files in the
directory will be removed until the second threashold or 90% setting
has been satisfied. Just put (or run) the script immediately before
any writes to the directory. If that is not possible you could add it
to cron and have it run periodically.
location=/media/usbdisk/
size=`df $location|grep sdc|awk '{ print $5 }'|sed -e s/%//`
cd $location
if [ $size -gt "98" ]
then
while [ $size -gt "90" ]
do
i=`find . -name "*" -exec ls -lrt {} \; |sort -k8,8 -k6M,6M -
k7,7n | grep '^-' | head -1 |awk '{ print $9 }'`
if [ -n $i ]
then
rm $i
fi
size=`df $location|grep sdc|awk '{ print $5 }'|sed -e s/%//`
done
fi
|
|
|
|
|