Questions[incremental-backup](server)

With btrfs hitting production in Oracle EL 14th this month (together with working fsck and scrubbing from Linux 3.2) I was thinking of redesigning my current backup solution to utilise it. Note that I'm thinking about doing it for small amounts of data, less than 10TB, that's fairly static (less than 1% changed daily). In short a SMB/SOHO backup solution.

What the backup should do:

1. do a LVM snapshot of ext[234]/XFS/JFS on the production server
2. rsync/transfer changed data to btrfs on backup server
3. snapshot the btrfs filesystem
4. drop old snapshots when free space is running low

Pros:

• All files easily available, no decompression or loop mounting needed
• Past snapshots also easily available...
• ... so I can share them as read-only Samba shares (with shadow copy support)
• Snapshots take minimal amount of space thanks to copy-on-write (snapshot without changes takes literally few KiB on disk)
• High backup consistency: checksums on files, scrubbing of all data and built-in redundancy

Questions:

• Is there some backup solution (in form of Bacula, BackupPC, etc.) that is, or can be easily made, aware of copy-on-write file system?
• Or will I need to use in-home rsync solution?
• What do people with ZFS boxes dedicated for backup do to backup their Linux machines?

I just spent hours building a new database server to replace 2 failed servers from a mysqldump file created a month ago, then used my server's bin-logs to bring that mysqldump up to date. Now that I have this new database server online, and have had it running for a few days, I need to complete the server set-up and re-implement a backup strategy.

I need to bring (at least) one MySQL Replication Slave online for the new server, as well as start creating usable backups for when things fail again.

Before I move further:

• I forgot (eep) to enable the bin-log on the new MySQL server before importing the old logical backup and executing the old bin logs (blame it on being awake for 24 hours at that point, I guess).
  
  
• My data consists of several databases, all which have a mix of MyISAM tables AND InnoDB tables.
  
  
• Also, along the same lines of enabling the bin logs on the new server, I forgot to turn on the innodb-file-per-table option in the my.cnf before importing all the data. The binary log is now enabled on the new MySQL server, but the inno file per table option is pretty much a non-fix until I can import another logical backup (I will probably end up bringing up a second slave, and promoting the first slave to master and ditching the current master - as the config just isn't where I need it to be).

So, the problem:
I need to take a backup of the MySQL server, but I can not take the new MySQL server offline. I need to continue taking writes, and serving reads with little to no downtime. This "little-to-no downtime" requirement is defined as less than 10 minutes. My data currently uses about 100GB of space on the server (mysql data files), and the logical backups are about 50GB (that's a lot of indexes..haha). I don't care of this backup is logical, or a copy of the data files from the MySQL data directory. I can create a logical backup off the slave after I get it online.

And, the question:
How would you create this needed backup? I know it's not easy, a lot of people will say it's not possible. But I refuse to believe that it isn't possible, there has to be a way to get it done.

A note about the server: It's running Ubuntu 10.04, MySQL 5.1.41, and the filesystem that the data is stored on is ext3. The server is running on the Rackspace Cloud, so the filesystem pretty much "is what it is" unless I can re-partition the root device and repartition it with another FS (XFS maybe?) to do a snapshot.

I've read about Perconas XtraBackup Tool but it only works for InnoDB tables. They have a MyISAM tool, but I don't really understand how it could (or even if it does) work in conjunction with XtraBackup to create a completely consistent backup.

I've read about mysqlhotcopy but it only works with MyISAM tables.

Obviously I know about mysqldump, but (also, obviously) the problem here is creating a consistent backup without locking the entire database server for the number of hours it will take to output that entire sql backup file.

Is it possible to restore a duplicity backup from a certain time in the past?

For example, if I'm making daily incremental backups, is there a way to restore a backup from three days ago?

I'm looking for a program to make incremental backups of one large file (3GB) on windows systems over internet.

Any idea? Thanks!

I'm trying to work up a backup strategy for some clients, and am leaning towards duplicity for remote backup (already use rdiff-backup for internal/on location backups).

Is it reasonable to want a full backup every so often? Since duplicity increments forward, each incremental backup is relying on the previous increment, and all are relying heavily on the last full backup. Should that become corrupt, bad things happen. A related question: Does Duplicity test the incremental backups for consistency?

Assuming I do want a full backup every so often, how efficiently does duplicity create that full backup? Can/does it check file signatures and copy unchanged data from previous full backups/increments? Basically creating a new 'full' archive transferring new/changed data and merging existing unchanged data?

Right now my concern is that running a full backup is needed, but the consistent large bandwidth use of full backups will make this unreasonable for some clients.