For benchmarking purposes I want to force Apache 2 to load each requested file from disk instead of loading it from a cache in memory. From what I have read doing a sync followed by
echo 3 > /proc/sys/vm/drop_caches
lets me drop linux's cache. A subsequent request for a particular file will then not be served from linux's cache, but further requests for the same file will be served from linux's cache again. That is so because /proc/sys/vm/drop_caches doesn't disable caching, it only discards what has been cached up to that moment. I could probably drop the cache before each and every request, but I'd prefer another solution. Is there anything else I can do to ensure that apache loads each requested file from disk?
Why I want to do this: I know that in normal operation caching is enabled. But the server is not serving small and frequently accessed files such as html pages, small images, etc. Instead it is serving files which are mostly a couple of megabytes in size from a very large set of files. These files are accessed pretty uniformly and therefore each individual file is accessed very rarely. Thus in normal operation I expect that most accesses will not cause a cache hit but require the file to be loaded from disk. I have several sample files which I want to access using apache's ab benchmark to measure how many transactions per second the server is able to serve. Unfortunately I believe the results I am getting are way too optimistic because of caching. Therefore I want to disable Linux's disk caching and any caching Apache might do itself.
Update: the answer given so far tells me how to disable Apache's own caching, but I am still wondering if there is a way to disable the caching done by the linux kernel.
I don't think you can disable all disk caching in Linux.
As a hack, you could keep running "sync; echo 3 > /proc/sys/vm/drop_caches" to flush almost anything that is cached in memory. From the console
would do the trick. In the above example nothing will remain cached by the kernel for more than a second, though it will have no effect on data held in memory by Apache or other processes. It may also not flush stuff from any memory-mapped files that are still open with portions locked.
If you only want nothing cached at the start of a test run, and don't care if it caches stuff during the tests, then you could just add a single call to "sync" and "echo 3 > /proc/sys/vm/drop_caches" at the start of your test run.
If your test involves scripts that access a database you will need to be able to tell your database back-end to not cache stuff in RAM between tests also.
You can disable the majority of caching in Apache by disabling the mod_cache module, commenting out the following lines in your configuration should do the trick:
LoadModule cache_moduleLoadModule disk_cache_moduleLoadModule mem_cache_moduleNo, it is not possible to disable the use of the buffer cache. There are ways to do this programatically, (opening the file in O_DIRECT for instance), but apache would have to be rewritten to do this.
At the core, it looks like you are trying to benchmark your IO subsystem for multiple streaming megabyte sized files. There are much better ways to do this without mixing the overhead of apache into the picture.
If you are determined to attempt this you could try writing a small program to malloc most of the physical memory in the machine to reduce the amount of disk cache available, however this would probably cause the server to page ruining your benchmark results (apaches multi process model will interact badly with this unless you set the
StartServers,MinServersandMaxServersto the same value to avoid process creation during your benchmark run)I might be terribly off-kilter here (so let me know if I'm making an incorrect assumption) but disk caching in Linux is known as swap, and uses a swapfile or swap partition.
Should disable disk caching.