Basic idea of what I'm trying to do: I'd like to setup a virtual staging cluster for a website i'm working on. Ideally I'd like to make a change, fire up and deploy it to the virtual staging cluster, then bring down a sample of data from production so that I can throughly test before doing a production rollout. When I was done testing on the virtual staging cluster I'd like cluster to go away (ie- remove/delete the VMs so I can also test from fresh installs)
My ultimate goal: My goal is to automate the process of setting up and tearing down the entire virtual cluster. Eg- like running a script on the server hosting the VMs that would create or delete all the VMs. Does anyone know if this could be done with existing virtualization solutions? I'm also open to discussion and tips about how you or your organization does this.
fyi- If I can get that far the virtual servers should be able to bootstrap themselves anew with PXE and debians FAI, and then puppet will finish the setup job.
VMWare offer a couple of products that might help with what you want to do:
Both VMware Server and VirtualBox have commandline tools. vmrun and VBoxManage respectively. With either product you can create scripts that create, start, stop, and remove machines (and much more).
For example (on VBox):
The above will start a VM called Windows.
For Virtual Box I'd look at: Using VBoxManage to control virtual machines
For VMware: vmrun command <- Warning, pdf
Yes, most virtualization solutions are scriptable nowadays.
With Xen, xen-tools, and FAI you might even get around without PXE (check the FAI wiki about integration with xen-tools).
But similar things should be possible with libvirt (which I'd recommend to use anyway, to get decoupling of the virtualization technology used) and KVM/XEN/QEMU/....
I've been thinking about doing something like for testing of FAI itself, and I#d use the crucible testing framework for it, it sounds appealing for such tasks http://sourceforge.net/projects/crucible/
Also, you might have a look at ganeti - it can also work with xen, kvm and maybe libvirt is also implemented in the meantime - if you need more VM's than a single machine can handle, this is very interesting because it'll helkp you also with things like file deployment and running commands on the virtual systems.
What virtualization platform are you using? VMware's Lab Manager product is working wonders at my workplace for setting up test-clusters and customer environments.