Since a few hours, my xen VM does not want to be created.
The VM is quite unstable so I need to destroy and create is quite often. This morning I was able to do such a destroy-create cycle. A few hours later, the destroy - create cycle failed with following error: The error
~: xm create /etc/xen/buildbox03.cfg -c Using config file "/etc/xen/buildbox03.cfg". Error: Creating domain failed: name=buildbox03
errorlogs in /var/log/xen/xend.log:
[2013-05-14 14:11:34 3735] DEBUG (XendDomainInfo:84) XendDomainInfo.create(['vm', ['name', 'buildbox03'], ['memory', '3072'], ['on_poweroff', 'destroy'], ['on_reboot', 'restart'], ['on_crash', 'restart'], ['vcpus', 3], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'], ['image', ['linux', ['kernel', '/boot/vmlinuz-2.6.34-1- netlog'], ['ip', ':1.2.3.4::::eth0:dhcp'], ['root', '/dev/xvda1 ro']]], ['device', ['vbd', ['uname', 'phy:/dev/ lvmpool/buildbox03-root'], ['dev', 'xvda1'], ['mode', 'w']]], ['device', ['vbd', ['uname', 'phy:/dev/lvmpool/ buildbox03-swap'], ['dev', 'xvda2'], ['mode', 'w']]], ['device', ['vif', ['bridge', 'br0'], ['mac', '00:16:3E:99:7F: A1']]], ['device', ['vif', ['bridge', 'br2'], ['mac', '00:16:3E:99:7F:A2']]]]) [2013-05-14 14:11:34 3735] DEBUG (XendDomainInfo:1618) XendDomainInfo.constructDomain [2013-05-14 14:11:34 3735] DEBUG (balloon:132) Balloon: 11813504 KiB free; need 2048; done. [2013-05-14 14:11:34 3735] ERROR (XendDomainInfo:440) VM start failed Traceback (most recent call last): File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 419, in start XendTask.log_progress(0, 30, self._constructDomain) File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendTask.py", line 209, in log_progress retval = func(*args, **kwds) File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 1656, in _constructDomain self.info['name_label']) VmError: Creating domain failed: name=buildbox03 [2013-05-14 14:11:34 3735] DEBUG (XendDomainInfo:1897) XendDomainInfo.destroy: domid=None [2013-05-14 14:11:34 3735] ERROR (XendDomainInfo:89) Domain construction failed Traceback (most recent call last): File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 87, in create vm.start() File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 419, in start XendTask.log_progress(0, 30, self._constructDomain) File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendTask.py", line 209, in log_progress retval = func(*args, **kwds) File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 1656, in _constructDomain self.info['name_label']) VmError: Creating domain failed: name=buildbox03 [2013-05-14 14:11:34 3735] DEBUG (XendDomainInfo:1897) XendDomainInfo.destroy: domid=None
buildbox03 config in /etc/xen/buildbox03.cfg:
kernel = '/boot/vmlinuz-2.6.34-1' memory = '3072' vcpus = 3 root = '/dev/xvda1 ro' disk = [ 'phy:/dev/lvmpool/buildbox03-root,xvda1,w', 'phy:/dev/lvmpool/buildbox03-swap,xvda2,w', ] name = 'buildbox03' dhcp = 'dhcp' vif = [ 'mac=xxxxxx,bridge=br0', 'mac=yyyyyyy,bridge=br2' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart'
Can you guide me to troubleshoot this problem a bit better?
Although we use XEN since 2007 I never did see an error like yours.
Your DomU-configuration looks good to me, so the error must be within the Hypervisor or the interaction with the Hypervisor of the Dom0 (
xend
orxenstore
).So some general advice:
For Problems with
xenstore
(typically: "hotplug-scripts not working"):Do
udevtrigger
orudevadm trigger
and wait for udev to settle (watch /var/log/messages). After that - try again. If it does still not work - udevtrigger again and retry. Further retries are useless, but this procedure covers about 90% of our xen-related problems.For problems with
xend
- your problem looks like one of it: Try to reload thexend
service:service xend reload
. If that fails try to restart ist:service xend restart
.Be careful here - there might be old hanging xend-processes afterwards that you have to kill manually.
xend restart/reload will not help with xenstore-problems (if you look into the init-script, you will see why: restart/reload dont restart/reload it).
I hope this helps in your case. Try it if your error reappears and comment on it...