I have two 120 GB disks in a software raid1 array (i.e. mirrored), showing as /dev/md2
. There's an ext3 filesystem on top of this, mounted at /
.
# uname -a
Linux svnserv 2.6.26-2-amd64 #1 SMP Sun Jun 21 04:47:08 UTC 2009 x86_64 GNU/Linux
Recently, writes to the disk started failing with "write error: No space left on device". df
showed that there's no more disk space. I removed some unused large files, approx. 5 GB, yet error persists.
df
shows this:
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md2 109G 104G 0 100% /
tmpfs 471M 0 471M 0% /lib/init/rw
udev 10M 100K 10M 1% /dev
tmpfs 471M 0 471M 0% /dev/shm
/dev/md0 44M 9.8M 32M 24% /boot
Note that 104 out of 109 GB is used, yet there's 0 GB available.
I'm not running out of inodes:
# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/md2 7208960 55273 7153687 1% /
mdadm
seems to be happy (both disks are active and in sync):
# mdadm --detail /dev/md2
/dev/md2:
Version : 00.90
Creation Time : Mon Jul 20 18:53:01 2009
Raid Level : raid1
Array Size : 115218112 (109.88 GiB 117.98 GB)
Used Dev Size : 115218112 (109.88 GiB 117.98 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Persistence : Superblock is persistent
Update Time : Thu Jul 8 14:34:08 2010
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
UUID : 1bb0fff8:38e78897:34693ac2:f3157c61
Events : 0.71
Number Major Minor RaidDevice State
0 22 3 0 active sync /dev/hdc3
1 22 67 1 active sync /dev/hdd3
How can I make the unused(?) space available?
Ah, the wonders of the SO/SF/SU trilogy: this question was kind of similar, but the answers didn't solve it. There was this comment however:
According to this article, default space reserved for root is 5% of disk size.
By issuing
tune2fs -m 0 /dev/md2
, the system now reports free space correctly (this system doesn't need root's reserved space).The problem was not related to RAID at all.