01-09-06 11:02 PM
Carla Tironi Farinati wrote:
>
> I'm trying to delete a very large directory full of small files (not
> more than 20k each). The directory weights around 40GB. I'm running
> Centos4.
>
> Now when I try "rm -rf mydir", the system hangs and do nothing.
> No change is displayed when I look for the free space.
>
> I also tried to do a "rm *" inside the directory to delete to see ...
With thousands of files the "rm *" will overflow the shell command
line buffer and fail.
With thousands of files "rm -rf mydir" probably needs to run a lot
longer to work.
Brute force method - This will take much longer but will run well:
find mydir \( \! -type d \) -print | xargs rm -f
find mydir -depth -print | xargs -1 rm -rf
rm -rf mydir
Since the first one will locate all non-directories and delete them
using a legal command line length, it will run for a very long time.
The second will run much faster. The third will run as fast as you
can hit the return.
Here's a bizzare case. I include it just in case the tree is so
deep that find runs out of virtual memory remembering how deep
it is:
Once I had a use try to install Clearcase on a workstation they
had the root password. (This is also a lesson why users should
never get root). They typed wrong and somehow convinced it
to try installing recursively. It made a chain of about 5-6 levels
of empty directory, cd'd to the bottom and then looped to create
the chain all over again. It started running and since the disk
access light was one the users went home and let it run. About
200 MB later the process finally ran out of virtual memory and failed.
By that time it created a single chain of directories so deep no
shell could cd that deep.
Clearly if I wrote a loop of moving the dir one level down up one
to flatten it the script would have run for a week. It had millions of
inodes in the chain .. to . each link. But if I wrote a loop to "cd *"
that would fail as well.
The good part was that -
while true ; do
cd *
done
ran for several minutes then failed. But when i tried to type "pwd"
it took minutes to start printing its name. Great -
cd ..
mv * /mountpt/lost+found
rm -rf /mountpt/lost+found/*
cd /mountput/lost+found
cd *
lather rinse repeat. Each attempt removed as many as their
names would fit in the shell's virtual memory and also moved the
rest up to an easily accessible location. However many dirs
the original chain was deep, that's how many times I had to
run that sequence. Prine pump by mooving to lost+found,
dive as deep as possible, move up one so it fits in VM, move
up to lost+found, attempt deletion, the "short"er chain suceeds.
[ Post a follow-up to this message ]
| 
