Standalone ESXi (4.1) host without any vCenter Server.
How to backup virtual machines as quickly and storage-friendly as possible?
I know I can access the ESXi console and use the standard Unix cp
command, but this has the downfall of copying the whole VMDK files, not only their actually used space; so, for a 30-GB VMDK of which only 1 GB is used, the backup would take 30 full GBs of space, and time accordingly.
And yes, I know about thin-provisioned virtual disks, but they tend to behave very badly when physically copied, and/or to blow up to their full provisioned size; also, they are not recommended for actual VM performance.
It is ok for me to shut down the VMs before backing them up (i.e. I don't need "live" backups); but I need a way to copy them around efficiently; and yes, a way to automate shutdown/startup when taking a backup would also help.
I only have ESXi; no Service Console, no vCenter Server... what's the best way to handle this task? Also, what about restores?
My preferred solution for this is to simply export them to an ovf or ova file using either the vSphere client or the command line ovftool.
In the vSphere Client, make sure the VM is off, then highlight it and go to File->Export->Export OVF Template. Then just follow the prompts.
Restoring is a piece of cake, just do the reverse (the menu option is "Deploy OVF template", I think).
To create a thin backup using ovftool
You may also wish to check out some of the options at http://www.virtuallyghetto.com/, I know these are very popular and I think there are some good choices for backups, although I haven't looked at any of them too recently.
I ended up writing a script which copies the VM configuration files and uses
vmkfstools -d
to clone the VMDKs while preserving the thin provisioning.For reference:
This requires the VM to be powered off and have no active snapshots.
I don't know if this fits the bill for you, but VM Explorer does a nice job of performing hot or cold backups of virtual machines. I believe that with ESXi 4.1 VM Explorer allows you to perform VM guest backups from one host to another host as well.
I would try to use a proven solution of some sort rather than roll your own. The reduced hassle, time, and risk will readily pay for itself even if you opt for a solution that isn't free. All of these issues you are concerned about are addressed in any modern backup solution for ESXi.
The solution that a client uses in their ESXi environment with good luck is Veeam. There is even a free edition that may work for your needs: https://www.veeam.com/virtual-machine-backup-solution-free.html
Ghetto VCB can do the backup while the machine is running. For the space you can use a deduplication+compression filesystem like lessfs on the backup server.
XSIBackup is an ESXi service that runs in the ESXi hypervisor OS and has a minimal footprint. In fact your users will not notice there's a backup taking place in the background.
It offers two different tools: XSIBackup-Free and XSIBackup-Pro.
XSIBackup-Free offers all the features present in the Pro version, such as:
The Pro version offers all of the above plus extended tools that will speed up your data transfers (XSIDiff), backup certification via checksum, quick file comparison methods and an SSH GUI (Graphical User Interface) that allows to configure all options from a wizard, speeding up the setup of backup jobs.
https://33hops.com/xsibackup-vmware-esxi-backup.html
I'm exporting to OVF format, too. It's fine because it compress on the fly the image with gzip.
But the licenses are gone, because with importing an OVF file into an ESXI a machine with new hardware is created.
Solution:
I backup (download) the file folder of the virtual machine without the big image and compress them.
Both - ovf folder and compressed copy of virtual image files (without big image) - are in the same folder.
Restoring:
-> Importing OVF into ESXI and upload the uncompressed config (from backuped file folder) -> Virtual Machine is fine again
ismail
cpio
can copy sparse files while preserving the "holes" in them.