Home / server / Questions / 310293
Accepted
Tomasz Grobelny
Asked: 2011-09-12 13:27:42 +0800 CST

ext2 dump/restore problem

  • 772

I am running a mailserver with maildir storage. This means that quite a lot of files are created and I have just run out of inodes. AFAIK there is no magic command to increase number of inodes on ext# filesystem (or am I wrong?) so I have to backup and restore the whole filesystem. But how do I do that? I tried creating another partition and do:

dump -f - -0 /vservers/mail | restore rf - -u -v

While this seems to work it takes much longer than I am willing to wait (it managed to create 500 empty directories in 2 hours before I stopped the process; strace showed that restore was calling lots of useless lseeks). Is there any other method to copy complete filesystem (including sockets, device files, owners, permissions, acls, etc)? Additional info: source fs is ext3, destination was ext4, filesystems are on lvm, the fs I want to move is root fs for vserver.

performance filesystems inode ext2 dump

3 Answers

  1. Best Answer

    My alternative suggestion for copying the filesystem follows. Keep in mind that the closest I've come to this problem is using find+xargs+rm to clear a maildir that had gone wild with useless junk, so you should see where this gets you after an hour or so.

    cd root_of_source ; find . -print0 | tar -c --null -T - -f - | tar -sxf - -C root_of_target

    The function of this construct is

    • Retrieve the list of files in their raw order
      • Which is why I use find instead of tar's default... I don't know that tar's default is bad, I just know that find's is good,
    • Pass that list to tar null terminated (so that any special characters are handled correctly).
    • Take the tar format output, and untar the result (without re-sorting the input (-s)) in the target directory.

    Regardless of the method you use:

    • If this data is starting and ending on the same physical disks, your performance will naturally suck a LOT compared to normal operations (lots of seeks between reading from the source and writing to the destination).
    • If you have some CPU available, a little compression shouldn't hurt, and may help Just add a 'gzip -c -1' stage between the tar commands, and a -z to the second tar.
    • 1

  2. Have you considered trying to use unionfs to union the existing filesystem with a new one, with writes going to the new filesystem?

    I've never used UnionFS myself, but what I have heard about it sounds like it could let you go live and start writing data to disk again without having to re-create the filesystem by unioning the existing filesystem read-only, with a new filesystem as the writable filesystem. There may be performance hits or other issues that make this a no-go, but if you're just looking for ideas and have some time while the dump is running, you can probably research this to a workable set of commands.

    • 0

  3. Other than dump | restore, you can use tar | tar, or just cp -ax or rsync to copy all of the files to the new fs. In my experience, dump | restore is the fastest method.

    For reference, on a rather old and slow machine, it takes me 35 minutes to duplicate an fs using dump | restore where the fs has 420,774 inodes using 7.8 GB of space.

    By comparison, it takes 61 minutes using tar | tar, and 64 minutes using cp -ax.

    A few months ago, I posted a patch to make dump faster, but it was after 0.4b44 was released, and there has not been another release yet. You can find the patch on the mailing list. Building 0.4b44 yourself with this patch applied may make a significant difference. For me, it reduced the time from 35 minutes to only 25. Feedback on the mailing list would be helpful.

    • 0