Home / server / Questions / 512195
Accepted
Korjavin Ivan
Asked: 2013-05-31 23:42:00 +0800 CST

Script for remove old backups

  • 772

I have a lot of backups, like:

backup-20130528_054504_mysql.tar.gz, 
bill_db_201305290130.BAK.bz2,
etc_20130412.tbz

This is full backups which done each day.

Before today, I have a cron task with a command line like this:

/usr/bin/find /media/backups -ctime +7 -delete

But it was the bad idea.

One server stopped do new backups (filesystem was full) more than 7 days ago, and all old backups was deleted by cron.

Probably I need a new script as replacement for this "find -ctime", which will hold some old backups anyway.

All my backup files have names {unique_prefix}YYYYMMDD...

I want something like :

  • one backup for each year before last year
  • one backup for each month of last year
  • 4 backup for each week of previous month
  • and 7 backups for last 7 days.

for each {unique_prefix}.

All this files are downloaded to my server by cron via rsync, ssh , etc..

I dislike reinvent the wheel, and pretty sure that is a typical task.

Please, point me to the solution.

backup

3 Answers

  1. A shell script can handle backups up to a certain point of complexity. You also have tools that leverage tar and rsync to accomplish even more convoluted tasks (rsnapshot, for example).

    However, it looks like you need a very fine-grained backup and retention policy that a shell script might not be well suited to handle unless it is unnecessarily complicated and the other tools mentioned do not provide.

    You should consider using a dedicated software for this task. There are many, but bacula and amanda come to mind.

    • 3
  2. Best Answer

    I disagree with dawud and think simple backups are the best.

    What you really need is not a smarter way of deleting things (though that wouldn't be a bad idea, I wrote something similar which simply keeps the last N backups), but proper monitoring of your filesystem so you'll notice it filling up well before it does and you can take action. That way you won't lose your data.

    • 2

  3. You can use time based policy (dangerous if you stop producing new backups, as all eventually be deleted):

    find . -type f -name 'app-*.tar.gz' -mtime +100 --delete

    Keeping last X items is safe:

    ls -t app-*.tar.gz | tail -n +100 | xargs --no-run-if-empty rm

    The best you can get with daily/weekly/monthly/yearly strategy which is not trivial to implement. You can try luck with rsnapshot and cron.

    There are libs that supports deletion of old snapshots, you need to adopt their conventions though:

    • 0