Home / server / Questions / 227437
Accepted
Nic
Asked: 2011-01-27 19:55:58 +0800 CST

How do you verify a restore?

  • 772

What tool(s) would you use to verify that a restored file structure is whole and complete? My environment is a Windows Server 2008 file server. (We use tape for backup, but that is inconsequential.)

I am specifically looking for a tool that will:

  • Record the names of all files and folders below a specified directory
  • Optionally calculate checksums of each file encountered
  • Save this index in a human-readable format
  • Compare the index against restored data and show differences

Some background: I recently had to replace the disks in our file server. The upgrade was scheduled to start 36 hours after the most recent full backup, so I created a differential backup. However, it turns out that one of our applications was clearing the archive bit on files saved to the server, so these were not included in the differential backup. I was unaware of this until my users reported some files as missing.

Aside from this, are there any other common methods for validating the integrity completeness of a restore? I am frequently told that testing backups by restoring them is the only way to know that backups are working, but how do you deal with the case where it works 99% correctly and the other 1% silently fails?

Update: Apparently I need to clarify a few things.

  • I do already use full backups when possible, but sometimes the situation calls for a differential backup. When that happens, I need to verify that every file in the original data is also in the restored data.
  • I am already using the "verify" feature in Backup Exec, but that only ensures that everything written to tape can be read back again.
  • I do conduct occasional spot-check restores to ensure that the backup media is intact.

I am already familiar with the common wisdom that "the best way to test a backup is to restore it." This is a necessary step, but it is NOT sufficient. Being able to restore the files you backed up does NOT guarantee that all the files you need were backed up in the first place. That is the problem I need solved.

windows-server-2008 backup-restoration verification

8 Answers

  1. Best Answer

    There are a variety of tools available on Linux which are well-suited to this task. You can use mount.cifs to mount Windows shared folders on a Linux host, or you could just run Cygwin right on the file server.

    Before starting the backup, use the find command to recursively iterate from a specified directory and write the results to a file. This listing can be saved along with the backup for future use.

    find /path/to/dir > list_before.txt

    If you want to have checksums calculated for every file, just pass the output through md5. This command only shows filenames because folders don't need hashes.

    find /path/to/dir -type f -print0 | xargs -0 md5 > md5_before.txt

    After restoring the backup, build another file list using the same command, then use diff to find differences between them. Ideally, this command should give no output.

    diff list_before.txt list_after.txt
    • 4

  2. First of all, enable the the 'verify' option in your chosen backup app and then stick to complete backups where-ever possible.

    You can use additional apps to perform all the actions you want, but they will take as long as the backup does. Maybe something to add to the weekend work-load of your servers?

    • 0

  3. Backup Exec (in recent versions) should verify after backup by default. Double check it though, should be a checkbox in the options.

    You might look at the "Write checksums to media" option to save checksums after each backup, and consider saving the job logs to compare from run to run. I don't know the format of these files, but you may be able to get file lists or at least size details to compare, as a starting point.

    • 0

  4. The best way to check a backup is to restore it. Anything else is a compromise - nothing wrong with compromises but you really do need to do restores of your data to test.

    In an ideal world you'd do a DR restore every 6 to 12 months, and restore random files on a more frequent basis, but any routine where you restore at least one server onto a virtual machine and check it boots afterwards is a great start.

    This is something you'd do in addition to any verification routine that the backup software itself has.

    • 0

  5. I use a combination of methods for backups. I use an on-line backup, as well as taking weekly images of my production servers. I do test restores on a monthly basis of random files such as SQL databases and attach them and verify they are functional.

    With my imaging, I do P2V backups of my servers using SCVMM into a big SAN. For DR testing I can bring them all up in a separate IP environment. If a server ever physically fails, I can bring up a VM of the server which are always less than a week old, and restore any discrepancies from the on-line backup. I also have a single XP machine joined to the domain that sits in that closed environment where I can test all my apps and email. I do this every 6 months or so to ensure a good DR environment.

    • 0

  6. Sorry, can't post a comment.)

    As far as i can tell (i'm not a windows guy), Nic's solution should work in Windows "natively" (just find and get UnixUtils for win32 or any similar package).

    You also can diff directories directly (optionally with trailing > difffile):

    diff -r /path/to/what-to-backup /path/to/restored-data
    • 0

  7. Not what you want to hear but I have the luxury of full 1:1 ratio reference environments for all my platforms for just such tests.

    • 0

  8. I would restore the files to a test location and use a tool like Winmerge:

    http://lifehacker.com/290657/compare-and-merge-files-and-folders-with-winmerge

    to compare them to the original source. There is also Windiff:

    http://www.computerperformance.co.uk/w2k3/utilities/windiff.htm

    I would also recommending backing up your valuable data three different ways, especially if you aren't verifying backups every day. I would suggest Backup Exec to tape, an offsite rsnapshot file backup, and disk based backup:

    http://backuppc.sourceforge.net/

    running locally. Try Backuppc, you'll thank me. When something goes wrong you'll appreciate the variety of options.

    • 0