Home / server / Questions / 370388
In Process
Vivek Sharma
Asked: 2012-03-17 02:11:27 +0800 CST

Not able to delete lot of files at once in linux

  • 772

I have run the following command, from inside of these log folders. But it looks like each folder will take infinite time.

find . -type f -exec rm -v {} \; 

rwxrwxr-x  2 root     root      77881344 Mar 16 03:06 logs.123  
drwxrwxr-x  2 root     root     105709568 Nov 14 20:09 logs_15Nov2011  
drwxrwxr-x  2 root     root       6852608 Aug  1  2011 logs2  
drwxrwxr-x  2 root     root     286191616 Nov  2 08:40 logs_2Nov  
drwxrwxr-x  2 root     root      25206784 Nov 10 04:04 logs_del  
drwxrwxr-x  4 root     root       2686976 Oct  6 01:56 logs_delete  
drwxrwxr-x  2 root     root          4096 May 11  2011 logsMay112011  
drwxrwxr-x  2 root     root      69087232 Aug 29  2011 logs_old  
drwxrwxr-x  7 root     root     382480384 May  9  2011 logs.old905  
drwxrwxr-x  2 root     root          4096 May 11  2011 logsTR1218

Any suggestion better and faster than this to remove all these logs?

linux logging bash

5 Answers

  1. GNU find has the -delete option, which is always safe:

    find . -type -f -delete

    As commented elsewhere, you may use xargs too, but be very careful how you use it.

    find . -type f -print0 | xargs -0 rm -vf
    • 6

  2. All the other answers assume that you want to keep the directories, but it's not clear from your original message that you do; moreover, even if you wanted to keep the structure, you'll need to remove and recreate the directories anyway, because the directory files have become very large - they won't shrink when you empty them, and their size will massively slow down operations in them in future.

    So have you considered just

    rm -rf log*
    • 2

  3. If you can manage the many errors when rm fails to delete directories, you can approach the problem from the other side

    for dir in $(find . -type d)
do
  rm $dir/*
done
    • 1

  4. Try

    find . -type f | xargs rm -vf
    • 0

  5. Other than reformating the partition, I do not see any faster way to delete all the files.

    The problem is that you have a huge amount of files in those directories (the size in the ls -l output that you provide is enormous !). For each rm, the filesystem must do the following:

    1. Queries the directory for inode number,
    2. Update the directory to remove the file from it,
    3. Decrements the link count of the inode (see hard link if you want to know more about link count),
    4. If the link count is zero, remove the data from disk.

    Even if you rm the directory, you only get rid of the step # 2 above becasue you still need to query the directory and decrement the link count. Querying the directory could be optimize to get more than one inode number at the time, but in all cases, for each file to delete, the filesystem must check the inode for the link count and update the inode (or mark it for deletion).

    In other word, this operation is O(n) on the number of files to delete, which seams very large in you case.

    If you cannot format your partition, which I guess is your case ;-), just launch the task in the background or in a screen and be patient.

    Another solution is to backup the files you want to keep, format the partition, and restore the files. Maybe it can be faster than deleting all the files, but is it more time consuming (the find...rm can be forgot in the background for many days if needed...).

    • 0