While troubleshooting a performance issue on a Debian Linux box running ext3 on software RAID 1 I redirected the output of a process writing useless temporary log files to /dev/null (ln -s /dev/null/ foo) which has reduced write operations by around 3MB/s. However the change seems to have also lead to a big increase in Write and Overall IOwait times which I'm at a loss to explain and would really appreciate some input on...
iostat: http://oi45.tinypic.com/5l2mx0.jpg
write latency (change applied at ~3am): http://oi46.tinypic.com/33be1kw.jpg
Update 18.04.13:
Write IOwait remains higher than before but I've applied the following all of which have been at least somewhat beneficial...
- Remounted the filesystem with noatime option.
- Changed to deadline ioscheduler.
- Made vm.swappiness 30 (host is home to ~1GB of database, a CMS and streaming audio).
- Turned readahead down to 256 down from 4096 for sda and sdb.
- Turned readahead up to 4096 from 256 for /dev/md2 (the main data partition).
Update 19.04.13:
- Converted the main filesystem from ext3 to ext4