Home / server / Questions / 532933
Accepted
DevOps
Asked: 2013-08-23 04:55:47 +0800 CST

How to increase deletion speed of several folder

  • 772

I have a script containing about 420k lines of "rm -rf" command like which where generated using a "find" statement. Each pdf's folder is containing between 1 and 30 files (no subfolder).

rm -rf /2012/128/211503/pdf
rm -rf /2012/128/212897/pdf
rm -rf /2012/128/211989/pdf
rm -rf /2012/128/211691/pdf
rm -rf /2012/128/212539/pdf
rm -rf /2012/218/358976/pdf
rm -rf /2012/218/358275/pdf
rm -rf /2012/218/358699/pdf

I'm searching how to increase the deletion speed of the script.

Currently, vmstat report only about (IO) wait time.

Platform is RHEL 5 deleting files on a RAID5/6 drive using ext3 and LVM.

I thought about splitting the script file into smaller files (like 10 files) in order to trigger several script in parallel but here I'm spotting a hardware speed limitation.

Would that be a good idea if the commitment of the deletion for the journalization taking time and could it take part of feature like NCQ ?

linux

1 Answers

  1. Best Answer

    If you're using find to generate the script you should take a look at the -delete action

    Delete files; true if removal succeeded. If the removal failed, an error message is issued. If -delete fails, find's exit status will be nonzero (when it eventually exits). Use of -delete automatically turns on the -depth option

    You could use split to break up the file into chunks. You may get some milage out of GNU Parallel too.

    • 1