I'm running into an issue when trying to reboot or shutdown my primary storage server at home. The shutdown process unmounts and stops most of the system until it hits systemd-udevd, then it hangs.
This what's displayed on the monitor:
...
[ OK ] Reached target Unmount All Filesystems.
[ OK ] Stopped Remount Root and Kernel File Systems.
[ OK ] Reached target Shutdown.
[ OK ] Reached target Final Step.
[ OK ] Finished Reboot.
[ OK ] Reached target Reboot.
[2706010.062757] systems-shutdown[1]: Waiting for process: systemd-udevd, systemd-udevd
[2706100.061724] systems-shutdown[1]: Waiting for process: systemd-udevd
[2706190.199550] shutdown[1]: Waiting for process: systemd-udevd
[2706280.198498] shutdown[1]: Waiting for process: systemd-udevd
[2706360.222337] sd-umoun[120154]: Failed to unmount /oldroot: Device or resource busy
[2706360.223343] sd-umoun[120155]: Failed to unmount /oldroot: Device or resource busy
[2706360.224401] sd-umoun[120156]: Failed to unmount /oldroot: Device or resource busy
[2706360.229444] shutdown[1]: Failed to finalize file systems, ignoring
At this point the system just sits there until I physically restart the machine.
This is an excerpt from /var/log/syslog that takes place around the time of the hang:
May 6 07:33:22 ubuntu-store systemd[1]: Stopped Initialize hardware monitoring sensors.
May 6 07:33:22 ubuntu-store systemd[1]: Stopping LVM event activation on device 8:16...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping LVM event activation on device 8:32...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping LVM event activation on device 8:48...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping LVM event activation on device 8:64...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping LVM event activation on device 8:80...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping Dispatcher daemon for systemd-networkd...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping PackageKit Daemon...
May 6 07:33:22 ubuntu-store systemd[1]: Stopping Authorization Manager...
May 6 08:05:50 ubuntu-store systemd-sysctl[433]: Not setting net/ipv4/conf/all/promote_secondaries (explicit setting exists).
May 6 08:05:50 ubuntu-store systemd-sysctl[433]: Not setting net/ipv4/conf/default/promote_secondaries (explicit setting exists).
The machine was manually restarted at 08:05:50.
A bit about the system:
- Ubuntu Server 20.04 with all updates
- Runs as a dedicated Samba server
- 5th-Gen Intel Core i5 w/16GB RAM
- 250GB SSD boot drive
- 25TB ZFS storage pool
lsb_release shows:
Distributor ID: Ubuntu
Description: Ubuntu 20.04.2 LTS
Release: 20.04
Codename: focal
While uname -r shows:
5.4.0-72-generic
The issue has only recently been observed (last month or so), as it doesn't get shut down very often unless there are security updates that require a reboot. One anomaly that I have yet to trace down is a load inconsistency. The CPU utilisation rarely exceeds 4%. The RAM usage rarely exceeds 9GB. The swap file, when used, might be a few dozen megabytes at most. The load, however, sits around 2.2. Running htop or glances does not reveal anything out of the ordinary. Not sure if this is evidence of a zombie process that isn't identified as such by the system.
As one would expect, rebooting the system returns everything to normal for a couple of days. However, as this is a server, I am not enamoured with the idea of having the machine reboot itself every Sunday night as though it were a Windows desktop from the 90s.
Any ideas on what might be causing the issue and/or how I might resolve it?
I was facing the same issue. I had to complete re write the ubuntu server on my sd card so that I could run it again. (I was working on raspberry pi 4 8gb RAM).
After I reproduced the error, I found that the a partition named writable. I saw that the partition was just gone just cause I rebooted the server.
As a temporary fix you could recreate that partition as writable and select the partition system as ext4 (linux file system).