So I've been trying to figure out this disk IO problem I have been having with my linode VPS. Over the last day or two I've just left
watch -n1 pidstat -d
running in a console window and the output looks like this:
Monitoring it over the last few days, I've noticed that my problem lies with the init, searchd, and mysql processes. Searchd is sphinx and all its indexes are on disk, so disk io there is inevitable (apparently).
What I can't understand is why the disk reads (kB_rd/s) for mysql refuse to stabilize and just keep going up. It started out at 154 yesterday and is up to what you see in that screen shot. but disk writes (kB_wr/s) have remained pretty constant the entire time.
My VPS only has 768MB RAM, my mysql db has a size of about 220MB and after running mysqltuner.pl and reading a bit about it, I've been advised to set my innodb_buffer_pool_size to 220MB but I simply cannot afford to do that ... I have it up to 150MB.
My question is twofold.
Why does the init process have that much disk reading to do?
Why is mysql doing so much disk reading?
Disabling the query cache based on this http://mysql.rjweb.org/doc.php/memory (Query cache section) and increasing innodb_buffer_pool size to 200MB solved the problem. Disk IO is under control but my vps now uses up a lot more swap than it did before (30-40MB vs 10-20MB) ... oddly enough the swap is being used by passenger application instances not mysql.