Home / server / Questions / 234382
Accepted
Fabio
Asked: 2011-02-12 06:22:30 +0800 CST

Find files containing a string on the whole filesystem

  • 772

I need to find all the instances of a given string in the whole filesystem, because I don't remember in which configuration files, script or any other programs I put it and I need to update that string with a new one.

I tried with the following command

`grep -nr 'needle' / --exclude-dir=.svn | mail [email protected] -s 'References on xxx'

If I run this command on a small directory it gives me the output I need in the form

/path1/:nn:line containing needle
/path2/:nn:line containing needle

where /path1 is the full path of the file, nn is the row containing the needle and last field is the content of the line.

However when I run the command on the root directory the grep process hang after a while. I run this script about 8 hours ago and even on a small filesystem (less than 5GB) it doesn't end and if I run top or ps the process seems sleeping

root 24909 0.0 0.1 3772 1520 pts/1 S+ Feb10 0:15 grep -nr needle / --exclude-dir=.svn

Why it doesn't end? Is there any better way to do this (it's a one time job, I don't need to execute this more than once)

Thanks.

Update: I found a working solution with find and xargs which seems to be working and uses less system resources than the find -exec solution. Here is my final command line:

find /{boot,etc,home,lib,lost+found,opt,root,sbin,usr,var} -type f -print0 | xargs -r0 grep -nr 'needle' | mail [email protected] -s 'References on xxx'

I used the /{dir1,dir2,...,dirn} syntax because I want the output lines with the fullpath including the leading slash, and in this way you can use the command without cd / i.e. from any directory.

find grep xargs

5 Answers

  1. Best Answer

    There are some files in the file system that aren't real files, but are instead hooks into the kernel. Some of those can be read from forever. Try

    grep foo /dev/zero

    to see this in action. Get ready to stop it with ctrl-C before it takes over the whole system.

    If I wanted to do what you're doing, I'd enumerate the subdirectories of / that I wanted scanned, and make sure I only checked text files, probably with

    cd /
find boot etc home lib lost+found media mnt opt root sbin tmp usr var -type f -exec grep needle {} /dev/null \;

    Note how the list does not contain /dev, /proc, /sys or /selinux.

    • 2

  2. You can try using find + xargs + grep for that, kinda:

    find /there -type f :MaybeSomeRestrictingFlagsLikeSizeNotBigger500MB_or_FS_type_if_u_know_its_exactly_on_EXT3_AndSoOn: -print0 | xargs -0r grep needle /dev/null

    (/dev/null makes grep printing file name even if only one file was found)

    • 2

  3. I don't know why it doesn't end but "fgrep -R" usually works in those cases.

    • 1

  4. You might want to exclude dev and proc from that command. I would do it with find instead and exclude special files:

    find . -type f -exec grep -H "My search string" '{}' \; -print
    • 0

  5. As some of the others pointed out, the error is occuring because you are using grep on some 'files' that will produce weird results. For example grep XXX /dev/zero will go on forever.

    I would first exclude the directories such as /dev/ and /proc/ from your search. Another way is to only search for 'files' and no other types.

    Also if your argument list in grep is too long, you will get this error:

    bash: /bin/grep: Argument list too long

    You can get around this with a loop, such as this:

    for i in `find -type f /`; do grep -H "sample string" $i; done

    Or by executing it from find, whatever floats your boat.

    find / -type f -exec grep -H "sample string" '{}' \; -print

    This should do a find and then grep each file. Although some have pointed out that it is inefficient to spawn grep each time you find a file, you will have to do something which will be 'inefficient' because you do literally have to search every file for that string.

    • 0