An Ubuntu LTS server runs xen with dom0 and one virtual machine. The server is freezing permanently on a number of seemingly unrelated operations, such as:
- Creation of a new file system with mkfs.ext3 on a LVM device. (this is consistent).
- Restart of xend via /etc/init.d/xend restart
- apt-get dist-upgrade on configuration phase of some fairly innocent stuff.
Also, yesterday I noticed that virtualized imagine had lost time synch and complained about backwards clock in dmesg.
Unfortunately, I don't have the screen shots on what happens actually on the console of the server (it is co-located).
I want to blame ram, but do You have other suggestions?
UPDATE: After further investigation, it appears that all those actions only kill network. When I visited the server in data center and logged onto console, I wasn't able to reach my router/gateway. How bizare.
for the network issue, xen works better if you don't let it set up the bridge...
for /etc/network/interfaces
/etc/xen/xend-config.sxp:
this way starting and stopping xen won't mess with your network interface.
Yeah, I'd be running a lengthy memtest first up, but there's a decent possibility it's something else hardwareish -- run a complete burnin of all system components, and monitor all your temperatures.
The clock thing is unrelated, Xen just can't keep good time. NTP forever.
That does look like hardware failure. Test the ram and also check the hdd for bad sectors. Also check the log files for any warnings.