Home / ubuntu / Questions / 193149
Accepted
black_puppydog
Asked: 2012-09-27 03:22:24 +0800 CST

Using duplicity directly, is it possible to have multiple backup plans?

  • 772

If I understand correctly, duplicity is the command line tool for creating backups and can be used stand-alone (i.e. no DejaDup involved) for creating backups.

Is it possible to set up multiple backup plans with different schedules and sources/targets? In particular I would like to:

  • regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.
  • back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)

I think this would just need some time setting up the different commands for the remote backup. The local backup could stay in DejaDup. But will duplicity be able to handle this? Or will it get confused with the different backups? Also: what will the performance for the remote backups be like? How will the actual diff+copy mechanism work in the remote case?

Why I ask this is just that with DejaDup I only get one backup plan, Back in Time only supports local copies (so I would have to mount the ftp locally and diff there which seems a waste of bandwidth) and rsnapshot uses hardlinks and is thus required to run on the target). but duplicity seems to be able to do what I want...

backup

2 Answers

  1. Best Answer

    In general, the answer is yes. Duplicity is highly flexible and you can back up different portions of your system differently. It's all about mastering the duplicity command line, however.

    • 2

  2. Is it possible to set up multiple backup plans with different schedules and sources/targets?

    Yes. In Duplicity you would run one command for each schedule. Duplicity doesn't have a built in scheduler, instead you use Cron. In it's simplest form a Duplicity backup scheduled by Cron would look like this:

    1. Make a bash script that contains the Duplicity command:

      #!/bin/bash
# Filename: myBackup.sh
duplicity --full-if-older-than 1M \
   --exclude-filelist excludeList.txt \
     / sftp://[email protected]/some_dir

      This script will make a full backup once every month. Otherwise it will make an incremental backup. It will make a backup of the root dir (/), but will exlude/ include according to the list below. It will save the backup at the sftp server (be sure to install ssh keys to make this work when the script is run by cron. If you run the script manually you have the choice to go without keys if you're prepared to enter your password each time you run it.

    2. Make an exclude list (exludeList.txt):

      **[Cc]ache*
**[Hh]istory*
**[Ss]ocket*
**[Tt]humb*
**[Tt]rash*
**.kvm
**.local/share/icons
**.rpmdb
**.thumbnails
**_NOBACKUP*
/home/user/VirtualBox VMs/**
/home/user/Downloads/**
+ /home/user
+ /var/www
+ /root
+ /etc
**

      In our Duplicity command we state that the source is /, and if you don't have an exclude file that means everything below the root dir is included in the backup.

      The double asterisk ** substitutes for every file name or path. First we list the files we want to exclude. These will be excluded even if they are contained in any of the dirs that we tell Duplicity to include. Then we tell which dirs to include (lines starting with +). In the end we tell Duplicity to exclude everything that we didn't mentions before, that is **.

    3. Schedule this with Cron by creating the following file in your /etc/cron.daily to run it once every day:

      #!/bin/bash
myBackup.sh

    regularly back up my complete home folder as I do now with deja-dup, i.e. whenever I connect my external drive and the backup is due.

    Tweak the backup command to your liking. To perform the backup when the external drive is connected you could for example:

    • Put a script in /etc/cron.hourly that checks the status of the backup (for example reads the date from a log file you create when you run your backup), if the backup has not been done today the script checks if the drive is mounted. If the drive is mounted backup is performed.

    • Write a service that runs when you plug your backup drive.

    back up different sets of folders to a remote server (Strato HiDrive, so ftp, smb or rsync) on a manual basis (ideally just a "trigger" command I issue) since I need to be in a place with fast internet connection to do this (I am speaking about ~200 GB here)

    This one is easy: Just create a script that runs Duplicity and uses an exclude file as described above.

    Good luck! Duplicity is a very versatile and stable software. If you're not used to writing bash scripts, it can be a bit challenging to use but I would definitely say it's worth a try for your needs.

    Let me know if there's any info you need that's missing in this answer.

    • 2