I have reports that I/O on a certain filesystem, namely /srv/data1, is intermittently horrific. I've seen it myself and it's true. For example, I run 'ls' /srv/data1 and the output takes 30+ seconds to appear. I run it again and this time, no delay. The periods of OK and horrific are short and cycle rather rapidly.
/srv/data1 is a 500GB OCFS2 on /dev/drbd1, which in turn is on systemA:/dev/md3 and systemB:/dev/md3. A and B are identical Dell R610s running SLES11. On both A and B /dev/md3 is a mirror of /dev/sdc1 and /dev/sdd1. And those are 500 GB SATA drives used for nothing else. The HDD controller is Dell's PERC 6/iR.
This is a busy, busy master Subversion server and the delays are breaking some processes that keep remote proxy servers in sync with this master. That's causing trouble for the development teams.
Any thoughts?
Thanks, Mike