Performed the latest Citrix XEN Server 6.2 updates, skipping anything related to 6.5. Everything seemed normal after fixing GrubConf.py except that a few random Windows Servers would not start. XenCenter failing machine log tab mentions "Error: This operation could not be performed because the specified virtual disk could not be found."
The virtual hard disk for the non-booting machines resides on a LVM over iSCSI disk. Confirmed that other virtual machines that use this same iSCSI disk were booting ok. Worried because we have had problems with our ethernet to SAS bridging machine and back-to-back extended power outages.
Received much better information from the commandline:
# xe vm-start vm='vmDoesNotStart' This operation cannot be performed because the specified VDI could not be found on the storage substrate sr: 34aabadd-ea86-ac22-2f3e-51fdadf41ebb (XenServer Tools) vdi: cdac77c7-011b-4184-8a47-361b6335d2e2 (Old version of xs-tools.iso)
Clicking eject next to 'DVD Drive 1:' fixed this and the servers start up. So turns out it had nothing to do with the "harddisks" but just "optical discs."