Home / server / Questions / 332577
Accepted
incognito2
Asked: 2011-11-19 14:38:43 +0800 CST

is it safe to running multiple instances of rsync together

  • 772

I'm wondering if its safe to use multiple rsync instances with the same command line to backup the same directory on the local server to the same directory on the remote server? basically if I ran the command below 3-4 times in parallel would it help speed things up? The recieving server is using rsyncd.

rsync -av /home/directory/ [email protected]::Home

backup rsync

3 Answers

  1. Best Answer

    No. You can break it up into multiple commands, excluding or specifically including directories, but running the same command several times will result in a mess (especially with the rsync temp files).

    • 14

  2. From version > 3.0.5 (maybe previous versions) you can use the --include, --exclude parameters.

    Taken the example from here you could do something along these lines:

    Assuming that directory_a has 2,000,000 files underneath it. and directory_b also has 2,000,000 files, running 2 separate rsync processes speed up the whole procedure in expense of CPU cycles.

    rsync -av --include="/directory_a*" --exclude="/*" --progress remote::/ /localdir/ >  /tmp/myoutputa.log &
rsync -av --include="/directory_b*" --exclude="/*" --progress remote::/ /localdir/ >  /tmp/myoutputb.log &

    According to the source this approach speeds up syncing by ~ 50%

    • 3

  3. 'Rsync' is already know for speeding up the transformation by transferring the selective and required part of that particular file or directory. Using the your mentioned command 3-4 times in parallel totally depends upon how much resources for your web hosting account has been granted, but one thing is certain that there would be more load created on server, so I would suggest you to do not go for it.

    Rsync features

    • 0