I'd like to run the following incomplete script weekly in as a cron job to backup my home directory to an external drive mounted as /mnt/backups
#!/bin/bash
#
TIMEDATE=$(date +%b-%d-%Y-%k:%M)
LASTBACKUP=pathToDirWithLastBackup
rsync -avr --numeric-ids --link-dest=$LASTBACKUP /home/myfiles /mnt/backups/myfiles$TIMEDATE
My first question is how do I correctly set LASTBACKUP to the the the directory in /backs most recently created?
Secondly, I'm under the impression that using --link-desk will mean that files in previous backups will not will not copied in later backups if they still exist but will rather symbolically link back to the originally copied files? However, I don't want to retain old files forever. What would be the best way to remove all the backups before a certain date without losing files that may think linked in those backups by currents backups? Basically I'm looking to merge all the files before a certain date to a certain date if that makes more sense than the way I initially framed the question :). Can --link-dest create hard links, and if so, just deleting previous directories wouldn't actually remove linked file?
Finally I'd like to add a line to my script that compresses each newly created backup folder (/mnt/backups/myfiles$TIMEDATE). Based on reading this question, I was wondering if I could just use this line
gzip --rsyncable /backups/myfiles$TIMEDATE
after I run rsync so that sequential rsync --link-dest executions would find already copied and compressed files?
I know that's a lot, so many thanks in advance for your help!!
You may wish to use a tool that automates the whole process for you, such as rsnapshot, which seems to implement what you are trying to do.
One question at a time please, I'm only going to answer the first question and you can post the others to another question:
In order to know what was the last backup dir, you simply add:
To the very end of the script, and then change your LASTBACKUP= to:
That way you always know your going to be using the last complete backup dir and not any other files/folders created in that directory, or backup folders that failed (which would create a mass of duplication)
You could use rsync.
and the second:
After creating the script to your needs add it to cron jobs.
add the following:
They cause make_snapshot.sh to be run every four hours on the hour and daily_snapshot_rotate.sh to be run every day at 13:00 (that is, 1:00 PM).
source: http://www.mikerubel.org/computers/rsync_snapshots/
If you want it to run hourly you would add a cron job for each hour.
Another possible option is using rsnapshot
Install rsnapshot (available in software center)
Configure rsnapshot and Specify Backup Source Directory
Open the /etc/rsnapshot.conf and uncomment the following lines.
Define your destination backup directories in /etc/rsnapshot.conf as shown below. In this example,
/home – source directory that should be backed-up localhost/ – destination directory where the backup will be stored. Please note that this directory will be created under /.snapshots/{internal.n}/ directory as shown in the last step.
nano /etc/rsnapshot.conf
backup /home/ localhost/
Test rsnapshot Configuration
Perform configuration test to make sure rsnapshot is setup properly and ready to perform linux rsync backup.
You can backup linux directories or files at various intervals. By default, the hourly and daily backups are configured.
Verify the hourly backup configuration.
Verify the daily rsnapshot cwrsync backup process is configured properly.
Once you’ve verified that the rsync hourly and daily backup configurations are setup properly in the rsnapshot cwrsync utility, it is time to set this puppy up in the crontab as shown below.
source: http://www.thegeekstuff.com/2009/08/tutorial-backup-linux-using-rsnapshot-rsync-utility/
So I will try to answer the second (and third) question:
--link-dest
creates additional hard links to existing files on the target filesystem. The number after the access permissions is the "link count", it tells you, how many links (think filenames) point to the file. For example:You can simply
rm -rf
old backups.rm
meansunlink
. It decreases the link count of a file. A filesystem will only overwrite files with a link count of zero. If you create additional files with gzip, they will just use extra space and you loose the advantage of the hard links.