Home / server / Questions / 584154
Accepted
zymhan
Asked: 2014-03-25 05:53:41 +0800 CST

Backing up rsnapshot directory with rsync copies entires backup directory

  • 772

I have a main backup server running rsnapshot, with ~2TB of backups stored on it. After the nightly backups, I copy the contents of the rsnapshot directory to an offsite server, using rsync -aH --delete /source /dest. However, this seems to copy the entire contents of the backup directory every night, as the offsite copy takes ~9 hours to complete.

I assume my rsync command doesn't have the right switches, but there may be something else I'm missing. Any ideas?

backup

3 Answers

  1. I believe you just copying the same files multiple times. Hope these details on how does the rsnapshot work will help resolve the problem.

    Here is the most important point:

    When rsnapshot daily is run, it will rotate all the daily.X directories, then copy the contents of hourly.5 into daily.0.

    hourly.0 will always contain the most recent snapshot, and daily.6 will always contain a snapshot from a week ago. Unless the files change between snapshots, the full backups are really just multiple hard links to the same files. Thus, if your /etc/passwd file doesn't change in a week, hourly.0/localhost/etc/passwd and daily.6/localhost/etc/passwd will literally be the same exact file.

    This is how rsnapshot can be so efficient on space. If the file changes at any point, the next backup will unlink the hard link in hourly.0, and replace it with a brand new file.

    • 4
  2. Best Answer

    The issue here is that if you want to keep a copy of the whole versioned backup on a remote location, you're going to have to handle the fact that rsnapshot rotates the directory names whenever it runs, so you'll end up with a fresh copy of the most recent backup being copied over every time.

    You can alter the way rsnapshot runs in terms of when it performs the copies and when it rotates (renames) the directories and you can also execute scripts at various points in rsnapshots execution cycle, like before or after a specific backup point is run.

    There's a helpful post here:

    https://stackoverflow.com/questions/10691522/mirror-rsnapshot-backup-directory

    Another approach is to get a copy of rsnapshot on your second machine and have it backup the contents of /snapshot_root/daily.0 AFTER your main back has been completed. This, in conjunction with the correct rsync flags will give you an incremental copy of your latest backups.

    • 2

  3. There is an option called --link-dest which does what you need. When this parameter points to your latest backup it will only copy files from src to dest if src is different from the data already in your last backup (referenced by --link-dest) ; if the data is the same it will create a symlink instead thereby giving you a whole versioned backup.

    I use this option to create daily snapshots locally on a separate disk, for remote backup you will need to modify it to ensure the symlinks will be created correctly on the remote system. I would recommend adding some rotation of old backups and an initial standard backup must be created to compare against.

    today=$(date "+%Y-%m-%d-%H-%M-%S")
rsync -a --link-dest=/localbackups/latest/ /data/ /localbackups/${today}
cd /localbackups/
rm -rf /localbackups/latest
ln -s $today latest
    • 1