Let's say you have a base server running several KVM guests. Now with regards to disk I/O caching, it would seem that the base server would cache the I/O in its memory, but wouldn't the guest also cache the same I/O in its memory pool also? If so, wouldn't this waste memory if the disk contents are cached twice? If this is the case, then what would be a good way to tune this behavior (i.e., either cache everything from the base server, or cache nothing from the base and only from withing the guests)? The only way I can think of is to allocate the guests with less memory that you normally would, set the swappiness parameter in the guests to 0 (so process memory doesn't get swapped out in favor of disk caching), so there is no extra memory in the guests that they can use for cache. This would then force all caching to happen in the base server. Or, is this an issue that is somehow solved automatically by KVM?
SnapOverflow
Yes, guests would use their memory for disk caching. Just like a non-virtual system would do.
Caching that may happen on the hypervisor depends on the type of disk interface you use for your guest and the type of cache policy you select. But whatever your choices are, caching on the hypervisor ould only happen for data the guest is wiling to commit to it's virtual drive, not with cached data.
This is why there are additional tunings available. For KVM guests, it is recommended to at least use
noopin the guest anddeadlineon the host. Deadline in the guest can sometimes yield better results, depending on the IO pattern.There are additional tunings available (dataplane, vhost-blk etc), each with it's own set of advantages and limitations.
Old, but still useful: http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/index.jsp?topic=%2Fliaat%2Fliaatbpstorage.htm