As the title says, I regularly download my Rackspace images for backup of my VPS, however I'm wondering if it's possible to run them locally on KVM (for example) so I can run tests on my application without affection production.
Almost a year to the day after the original question, I think I have a good procedure set up for downloading Rackspace images and running them locally, under Xen running on Ubuntu Precise.
I'll write down the steps here so there's an easy recipe for you to follow if you want to try it out. The steps work for linux/Ubuntu domU's at least, for others YMMV.
Go to your servers page on the rackspace cloud (manage.rackspacecloud.com -> Hosting -> Cloud servers) and select the one you want to run locally. Go to the Images tab for that server. Create an on-demand image. (Depending on the time the server was first created the on-demand image will either go directly to cloud files or be stored with the server. If the image is stored with the server, you have to move it to cloud files after creation.)
Check the name of the image in cloud files. Go to Hosting -> Cloud Files and open the container cloudservers. There should be the file there named like myservername_YYYYMMDD_XXXXX_cloudserverZZZZZ.tar.gz.0, and that's the file you want.
Download the image to your server. The easiest way to do this on Ubuntu 12.04 is to use swift. To install, just sudo apt-get install swift.
Then run it like swift -A https://auth.api.rackspacecloud.com/v1.0 -U username -K API_key download cloudservers myservername_YYYYMMDD_XXXXX_cloudserverZZZZZ.tar.gz.0
When the download is done, unpack the downloaded file (tar zxf myservername_YYYYMMDD_XXXXX_cloudserverZZZZZ.tar.gz.0). When it's done you'll have gotten the following files:
image
-> image.vhd
-> manifest.ovf
-> snap.vhd
image.vhd is the one we're interested in, as it contains your root partition from the cloudserver.
To make it easier to work with the image file, we'll convert it to a raw disk image. This is done using qemu-img (if it's not available on your system, do sudo apt-get install qemu-utils).
To convert the image.vhd to something that's easier to work with, do qemu-img convert -p image.vhd disk.img
Check the disk image using fdisk. (NOTE: I've only been working with 20 GB large images from rackspace, and my notes are about them. If you use another size, you'll have to adjust some steps accordingly!)
fdisk -l disk.img should give something like the following:
Disk sda1.img: 20.4 GB, 20400758784 bytes
255 heads, 63 sectors/track, 2480 cylinders, total 39845232 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Device Boot Start End Blocks Id System
disk.img1 * 2048 39845887 19921920 83 Linux
Note the start offset for the first partition, and the sector size.
(Optional) Check that the disk image is ok. For this, we mount it loopback and simply see that it works.
mkdir tmp
mount -o loop,offset=1048576 disk.img tmp/
ls -la tmp
(The offset above is start for the first partition * sector size)
This gives you an indication that the disk is ok. Then unmount the image. umount tmp
Create logical and swap volumes for your new DomU. lvcreate -L20G -n myservername-disk vg0 and lvcreate -L2G -Cy -n myservername-swap vg0. Also initialize the swap disk, mkswap /dev/vg0/myservername-swap.
Copy your disk image to the new logical volume. dd if=disk.img of=/dev/vg0/myservername-disk bs=4096 skip=256. The bs and skip arguments are there to get the correct offset for your partition. I could go with bs=512 skip=2048, of course, but making the block size larger makes copying much faster. (And 512*2048 = 4096*256, so it does not matter for the offset.)
Check the filesystem on your logical volume (and fix any errors), resize the fs to fit the lv, and check again.
Check the virtual disk settings in your virtual machine. This will be needed for your DomU config file.
Mount the logical volume. mount /dev/vg0/myservername-disk tmp. Then check the fstab file, and (if it exists) the grub menu.lst file.
more tmp/etc/fstab
more tmp/boot/grub/menu.lst
Note the disk names for root and swap!
Update the network settings for your DomU. With the virtual disk still mounted, edit the interfaces file to something that suits your local environment. vi tmp/etc/network/interfaces.
I've usually changed mine from something like:
# The loopback network interface
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 184.106.XXX.YYY
netmask 255.255.XXX.YYY
gateway 184.106.XXX.YYY
dns-nameservers 173.203.XXX.YYY 173.203.XXX.YYY
auto eth1
iface eth1 inet static
address 10.179.XXX.YYY
netmask 255.255.XXX.YYY
up route add -net 10.176.XXX.YYY0 netmask 255.240.XXX.YYY gw 10.179.XXX.YYY
down route del -net 10.176.XXX.YYY netmask 255.240.XXX.YYY gw 10.179.XXX.YYY
up route add -net 10.191.XXX.YYY netmask 255.255.XXX.YYY gw 10.179.XXX.YYY
down route del -net 10.191.XXX.YYY netmask 255.255.XXX.YYY gw 10.179.XXX.YYY
(for rackspace's environment) to:
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
for just using dhcp and one virtual network interface and my local server.
After these changes, we should be done. Unmount the virtual disk, umount tmp.
Create a xen config file for your imported DomU, vi /etc/xen/myservername.cfg, and put in something like this:
name = "myservername"
bootloader = '/usr/lib/xen-default/bin/pygrub'
#uncomment the following if you want to boot into single user mode
#extra = " single"
vcpus = '2'
memory = '512'
#
# Disk device(s). (Use the correct mappings from step 12.)
#
disk = [
'phy:/dev/vg0/myservername-disk,xvda1,w',
'phy:/dev/vg0/myservername-swap,xvda2,w',
]
#
# Networking
#
vif = [ 'bridge=xenbr0' ]
#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
Again, note that you should use the same device names for the virtual devices as the ones you discovered in step 12.
Now everything should be ready! To verify, create the domU and attach a console by running xm create -c /etc/xen/myservername.cfg
Almost a year to the day after the original question, I think I have a good procedure set up for downloading Rackspace images and running them locally, under Xen running on Ubuntu Precise.
I'll write down the steps here so there's an easy recipe for you to follow if you want to try it out. The steps work for linux/Ubuntu domU's at least, for others YMMV.
(These instructions are also posted on our blog at http://www.everichon.com/blog/2013/05/29/running-rackspace-images-locally/)
Install Ubuntu precise (12.04 LTS), and follow the steps mentioned in http://blog.heidt.biz/blog/2012/09/17/setting-up-ubuntu-12-dot-04-precise-pangolin-as-xen-dom0/ to set up your server as a Xen Dom0. At least make sure to have a running 12.04 with xen and xe-tools installaed, and a LVM volume group to use for virtual disks for your DomU's.
Go to your servers page on the rackspace cloud (
manage.rackspacecloud.com -> Hosting -> Cloud servers) and select the one you want to run locally. Go to theImagestab for that server. Create an on-demand image. (Depending on the time the server was first created the on-demand image will either go directly to cloud files or be stored with the server. If the image is stored with the server, you have to move it to cloud files after creation.)Check the name of the image in cloud files. Go to
Hosting -> Cloud Filesand open the containercloudservers. There should be the file there named likemyservername_YYYYMMDD_XXXXX_cloudserverZZZZZ.tar.gz.0, and that's the file you want.Download the image to your server. The easiest way to do this on Ubuntu 12.04 is to use
swift. To install, justsudo apt-get install swift. Then run it likeswift -A https://auth.api.rackspacecloud.com/v1.0 -U username -K API_key download cloudservers myservername_YYYYMMDD_XXXXX_cloudserverZZZZZ.tar.gz.0When the download is done, unpack the downloaded file (
tar zxf myservername_YYYYMMDD_XXXXX_cloudserverZZZZZ.tar.gz.0). When it's done you'll have gotten the following files:image.vhdis the one we're interested in, as it contains your root partition from the cloudserver.To make it easier to work with the image file, we'll convert it to a raw disk image. This is done using
qemu-img(if it's not available on your system, dosudo apt-get install qemu-utils). To convert the image.vhd to something that's easier to work with, doqemu-img convert -p image.vhd disk.imgCheck the disk image using fdisk. (NOTE: I've only been working with 20 GB large images from rackspace, and my notes are about them. If you use another size, you'll have to adjust some steps accordingly!)
fdisk -l disk.imgshould give something like the following:Note the start offset for the first partition, and the sector size.
(Optional) Check that the disk image is ok. For this, we mount it loopback and simply see that it works.
This gives you an indication that the disk is ok. Then unmount the image.
umount tmpCreate logical and swap volumes for your new DomU.
lvcreate -L20G -n myservername-disk vg0andlvcreate -L2G -Cy -n myservername-swap vg0. Also initialize the swap disk,mkswap /dev/vg0/myservername-swap.Copy your disk image to the new logical volume.
dd if=disk.img of=/dev/vg0/myservername-disk bs=4096 skip=256. Thebsandskiparguments are there to get the correct offset for your partition. I could go withbs=512 skip=2048, of course, but making the block size larger makes copying much faster. (And 512*2048 = 4096*256, so it does not matter for the offset.)Check the filesystem on your logical volume (and fix any errors), resize the fs to fit the lv, and check again.
Check the virtual disk settings in your virtual machine. This will be needed for your DomU config file. Mount the logical volume.
mount /dev/vg0/myservername-disk tmp. Then check the fstab file, and (if it exists) the grubmenu.lstfile.Note the disk names for root and swap!
Update the network settings for your DomU. With the virtual disk still mounted, edit the
interfacesfile to something that suits your local environment.vi tmp/etc/network/interfaces. I've usually changed mine from something like:(for rackspace's environment) to:
for just using dhcp and one virtual network interface and my local server. After these changes, we should be done. Unmount the virtual disk,
umount tmp.Create a xen config file for your imported DomU,
vi /etc/xen/myservername.cfg, and put in something like this:Again, note that you should use the same device names for the virtual devices as the ones you discovered in step 12.
Now everything should be ready! To verify, create the domU and attach a console by running
xm create -c /etc/xen/myservername.cfgYes, you can run them, the hypervisor they use is Xen IIRC. You are also able to download and install Openstack and have your own private cloud.
Here is good link on "Run Rackspace Cloud Server locally on Hyper-V"
http://www.sudonym.com/375/run-rackspace-cloud-server-locally-on-hyper-v