Home / server / Questions / 7969
Accepted
joeforker
Asked: 2009-05-14 13:12:27 +0800 CST

Is there a working Linux backup solution that uses inotify?

  • 772

It takes forever to back up. Before we can trust btrfs or ZFS to backup incremental snapshots, wouldn't it be nice if there was a daemon that used inotify to keep track of which files had actually changed so backups would run more quickly? Where is this program?

How do I backup my Linux box without having to crawl the whole filesystem every time? I would like a program that would detect the new or changed photos, source code, etc. and queue them to be copied over to my NAS.

backup linux inotify

7 Answers

  1. Best Answer

    I answered my own question with "yum search inotify". It's called lsyncd and it's hosted on google code.

    Unfortunately it looks like it always runs a full rsync first, so it still wouldn't help me if my computer was not turned on for more than 14 hours at a time.

    Lsyncd uses rsync to synchronize local directories with a remote machine running rsyncd. Lsyncd watches multiple directories trees through inotify. The first step after adding the watches is to rsync all directories with the remote host, and then sync single file by collecting the inotify events. So lsyncd is a light-weight live mirror solution that should be easy to install and use while blending well with your system. See lsyncd --help for detailed command line options.

    • 11

  2. There's a new system called fsnotify that's designed to solve the deficiencies of inotify which was introduced to solve the problems of dnotify. fsnotify lets you watch an entire filesystem without much fuss. Hopefully fsnotify will help solve all our future Linux backup problems.

    • 3

  3. You can hack something with incron. 

     /path1    IN_CLOSE_WRITE     rsync  -au $@/$#  backuphhost:/path

    • 2

  4. Based on research (not testing) it seems like inotify can't handle the very large number of files on most systems and/or is very slow doing so. The thread at http://www.pubbs.net/kernel/200905/109416/ was the most useful, It pointed to a new linux feature that seems to be in or headed towards mainline, fsnotify, that is in linux 2.6.31 and later.

    • 2

  5. Lsyncd syncs the whole watched tree on startup, because for 99% of cases this is the sensible thing to do. You do want the directory on the target host you have on the local host, otherwise syncs might fail and you want to sync things you missed while turned off. However, if you know what you are doing, you can turn off startup syncing, just set sync{..., startup=false} in the Lsyncd config file.

    regarding inotify, its not the number of files but the number of directories that eat up ressources. One directory is one watch, regardless how many files it contains.

    fanotify building on fsnotify like inotify looked very promising for people watching tons of directories, but currently as of Linux 2.6.37 fanotify does not report rename (move) events at all, making it unusable for a job like this :-(

    • 2

  6. I have spent 6 months looking for the best solution to perform what you're trying to do; efficiently backup to a NAS. After the initial sync, all else is smooth as butter. The latest version of Lsyncd works quite well. I've documented what I've done in the link below. Just substitute your folder values. Hope this helps:

    https://docs.google.com/document/d/1XpqM5h5YMwuQqzdknyDDnjcQVYGjAsyAxfYprqSnhd0/edit

    • 1

  7. there 's a new system used in asia,called sersync which will instead of inotify-tool+rsync resolution http://code.google.com/p/sersync/ it is very easy to use.

    • 0