Home / server / Questions / 499937
In Process
Andy Lester
Asked: 2013-04-17 10:56:10 +0800 CST

Can I get tar to not use the filesystem cache when reading files?

  • 772

We have a cronned job on a production web server that requires tarring up files from across the filesystem. It runs for about 105 minutes, and during that time, Apache starts swapping because all that reading of a few gigs of files has expanded the amount of memory for the filesystem cache.

The data that tar is reading doesn't need to live in the filesystem cache. It's going into the tarfile and that's it. Is there something where we can say like tar cvf whatever/ --no-system-file-buffer-reads? My reading of man tar doesn't turn up anything that looks like it would help.

You don't have to point out that this can be seen as a "Doc, it hurts when I go like that! So don't go like that" situation. Somehow we need to dump files into a tar file, so not going like that isn't an option.

linux

1 Answers

  1. I think this has been answered here:

    There is the nocache utility, which can prepended to a command like ionice and nice. It works by preloading a library which adds posix_fadvise with the POSIX_FADV_DONTNEED flag to any open calls.

    In simple terms, it advises the kernel that caching is not needed for that particular file; the kernel will then normally not cache the file. See here for the technical details.

    It does wonders for any huge copy jobs, e. g. if you want to backup a multi terabyte disk in the background with the least possible impact on you running system, you can do something along nice -n19 ionice -c3 nocache cp -a /vol /vol2.

    • 12