I have filesystem problems on a machine running Ubuntu 8.04 machine in a colocation facility. What was once a directory now looks like this:
b--xr-SrwT 52 825452087 876163170 56, 97 2021-10-10 05:43 58
This was cache data, so I don't mind the data loss, I just want to fix the filesystem. The problem is, this is the root filesystem, so I can't unmount it to run fsck.
So my question is: How do I run fsck on the root filesystem without having to drive all the way to the colocation center?
it's also a good idea on debian and debian-derivatives like ubuntu to edit /etc/default/rcS on remote servers and set "FSCKFIX=yes"
that adds "-y" to the boot time fsck, so it doesn't risk the remote server being stuck waiting for someone to login at the console and run fsck.
also, just in case something like happens again, it's worthwhile having a rescue partition that you can boot (e.g. temporarily set grub's default), ssh into, and run fsck on your real rootfs. if you don't currently have a spare partition free for this, you can shrink a swap partition to give yourself enough space to make a rescue partition (which you can populate with debootstrap).
and if you can't use a swap partition, you can set up a grub entry to boot into a live cd image (using the kernel and initrd from the iso)...but you'll have to modify the initrd filesystem to have your correct IP address etc and make sure that sshd is running. clonezilla, gparted or systemrescuecd would make good live systems to use as a base for this. your /boot partition/directory needs to be big enough to hold these files.
Should force a fsck of all disks in /etc/fstab on reboot. link: man shutdown
Alternatively, if you want a fsck on every reboot: