Home / server / Questions / 998769
In Process
Volodymyr Melnyk
Asked: 2020-01-14 12:34:09 +0800 CST

I've killed the processes that held the deleted files, but a filesystem is still full

  • 772

I have a host that runs CentOS-6 with Linux 2.6.32-696.18.7.el6.x86_64 (yes, pretty old) and I ran out of disk space on the root filesystem.

The filesystem's size is 16G, du shows about 3G being used by files. Of course, I ran lsof +L1, found processes that held the big log-files that had been unlinked before (yes, there were a couple of such processes, so I killed them), but the filesystem is still full and anything can write to it (no, inodes are fine, there are no available blocks).

lsof +L1 still shows some files, but these files don't occupy too much size, so I'm pretty sure I can ignore that:

COMMAND     PID      USER   FD   TYPE DEVICE SIZE/OFF NLINK   NODE NAME
init          1      root    9r   REG  253,0 10406312     0 131692 /var/lib/sss/mc/initgroups (deleted)
pickup     2761   postfix    8r   REG  253,0 10406312     0 134123 /var/lib/sss/mc/initgroups (deleted)
dbus-daem  5232      dbus    4r   REG  253,0 10406312     0 131692 /var/lib/sss/mc/initgroups (deleted)
hald       5271 haldaemon   19r   REG  253,0 10406312     0 131692 /var/lib/sss/mc/initgroups (deleted)
ntpd       5528       ntp    4r   REG  253,0 10406312     0 131730 /var/lib/sss/mc/initgroups (deleted)
qmgr       5631   postfix    8r   REG  253,0 10406312     0 131730 /var/lib/sss/mc/initgroups (deleted)
crond      5655      root    8r   REG  253,0  8406312     0 131792 /var/lib/sss/mc/passwd (deleted)
sssd       9042      root   16r   REG  253,0 10406312     0 134123 /var/lib/sss/mc/initgroups (deleted)
sssd_be    9045      root   18r   REG  253,0 10406312     0 134123 /var/lib/sss/mc/initgroups (deleted)
qemu-kvm   9606      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  16799      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  17419      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
polkitd   28420      root    8r   REG  253,0 10406312     0 134123 /var/lib/sss/mc/initgroups (deleted)
qemu-kvm  28802      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  30171      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  32110      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  32419      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  32563      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  34762      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  37823      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  39088      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
qemu-kvm  41613      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)
nrpe      45821      nrpe    4r   REG  253,0 10406312     0 134123 /var/lib/sss/mc/initgroups (deleted)
sshd      52715      root    4w   REG  253,0 10406312     0 134123 /var/lib/sss/mc/initgroups (deleted)
qemu-kvm  55329      root  txt    REG  253,0  4056888     0 397695 /usr/libexec/qemu-kvm (deleted)

What else should I try to do (I'm pretty sure that reboot will help, but I'd like to avoid doing that)?

Thanks!

linux

0 Answers