This is how / should be mounted
mount
/dev/md0 on / type ext3 (rw,errors=remount-ro)
but in fact, it is mounted read-only because of unknown reason so far for me.
cat /proc/mounts
/dev/md0 / ext3 ro,relatime,errors=remount-ro,data=ordered 0 0
:/var/log# echo aaa > /tmp/ddd
bash: /tmp/ddd: Read-only file system
As ext3 has errors=remount-ro flag, the possible reason is that the filesystem has errors. When I checked syslog/messages/dmesg I did not find any related information (no disk failures, filesystem errors etc.). This server does not have any iLo or so.
So what do you recommend, how to recover the root filesystem without working physically on the server.
Update:
:/var/log# mount -o remount,rw /
mount: cannot remount block device /dev/md0 read-write, is write-protected
You will probably need to reboot. You might not have filesystem errors: remounting the filesystem read-only is intended to protect it against problems with the storage hardware: hba cards, cables, storage arrays and so on; and usually is an overreaction.
That said, the closer linux is to the disks, the more likely it is you have a real hardware problem.