Home / server / Questions / 124883
Accepted
nickd
Asked: 2010-03-23 02:55:28 +0800 CST

Directory backed up with rsync is much bigger than source

  • 772

We backup a set of virtual machines to an external USB drive using rsync -a. The source directory is 145G as reported by du -sh, but the target is reporting 181G.

The two file systems are ext3 and the block size is the same, so can someone explain what the discrepancy is?

rsync

4 Answers

  1. Others already told about sparse files, but there is another thing: hard links. Hard links – multiple names for the file (and space on the disk) are often used on system partitions (e.g. for multiple shell commands implemented in the same binary) and they are not handled specially by rsync with the '-a' option only. So, e.g. a file with four hard links will be stored as four separate files.

    Try using rsync -aH.

    • 7
  2. Best Answer

    As Dennis mentioned, it seems to be a sparse file issue. An example of that can be:

    $ dd if=/dev/zero of=sparse.txt count=0 seek=1000
0+0 records in

$ du sparse.txt
0   sparse.txt

$ ls -l sparse.txt
-rw-r--r-- 1 user user 512000 2010-03-22 11:54 sparse.txt

    As you can see du reports how many blocks are actually used, while ls shows how big the file is supposed to be.

    • 6 
  3. -S, --sparse
              Try to handle sparse files efficiently  so  they  take  up  less
              space on the destination.  Conflicts with --inplace because it's
              not possible to overwrite data in a sparse fashion.

              NOTE: Don't use this option when the destination  is  a  Solaris
              “tmpfs”  filesystem.  It  doesn't seem to handle seeks over null
              regions correctly and ends up corrupting the files.
    • 4

  4. With the rsync script, are you deleting the items that exist on the destination and are no longer on the target? If you're looking to have an identical copy on both sides, you would need a "--delete" flag in your rsync routine.

    "rsync -a --delete /source/ /destination/"

    You can also inject "-n and -P" into the string to provide a dry-run and progress indicator, respectively, to show you what would happen with the "--delete" option.

    • 1