I recently set up a machine with Ubuntu Server to host game servers. I installed a backup plugin for each game server that creates frequent backups of game world files in a particular folder on the machine. I also established a cron
task to automatically copy those backups to my Dropbox folder every night using rsync with the -a
option.
After a few months my Dropbox account reached its storage limit and I realized I would not be able to keep so many backups, so I configured the game server backup plugin to not retain so many backups, then waited a few days to see if it would delete the older backups as it is scheduled to do on a weekly basis. The backup plugin eventually did its job and deleted the older backups, so I was expecting the rsync
cron task to subsequently delete the older backups from my Dropbox folder to match the source folder, but it has not done so. So I have a couple of questions:
By default, does
rsync
only add files to the destination folder that have been added to the source folder and change files that have been changed in the source folder but NOT delete files that were deleted from the source folder?If that is the case, what is the best way to make
rsync
do this? I want the destination folder to perfectly reflect the source folder, and that means deleting any files that have been deleted from the source folder.
I see some options listed in the manual page for rsync
that might do the trick, but since I'm not familiar with.
To delete files in the target, add the
--delete
option to your command. For example:NB:
-avh
is for--archive --verbose --human-readable
The rsync command wont delete any file while you use some of its options
delete
in that command. So if any file or folder added in source, it'll be synced to target without any deletion.I suggest you to use
rsync
for make backup from source files and usefind ... rm
for deletion files for period of time or size of files:The above code block, make a backup from source and then delete every files which last modified time are more than 2 month.
UPDATE
As I find that the
delete
options are just for TARGET that if some files are removed from source,rsync --delete
remove them from TARGET. And thedelete
option byafter
andbefore
, as mentioned in its man page:Means that:
Means that:
NOTE: The
--delete-{before/after}
implement just in TARGET.The command
takes care to always sync back changes done locally to the remote. That means it takes care to synchronize local and remote such that
The parameters explained:
-a
archive mode-v
increase verbosity-h
output numbers in a human-readable format-n
dry run, perform a trial run with no changes made. Always use this flag initially to prevent data loss. When you are happy, remove it.Let the results speak for themselves:
Create 2 directories
Create files in them
See what has been created (before rsync):
calling rsync:
See the result (after rsync):
As you can see, the file remote/remote_only has been deleted, the file local/local_only has been synchronized.
This command will copy increment data and keep it in sync with remote server.
rsync -avWe ssh --delete-before (source) root@localhost:(destination) rsync -avW --delete-before -e ssh (source) root@localhost:(destination)
Example:
If there are any errors during an rsync scync, rsync will not properly delete the files it should have, even if you used
--delete
,--delete-after
, or--delete-before
.This is why it is important to address rsync errors.
Most of my errors were due to using the
--perms
option while syncing with a Non-Linux file system. When I replaced--perms
with--no-perms
, those errors went away and then deleting worked.--perms
is ok when you are syncing from a Linux file system to a another Linux file system, but if you're syncing from Linux to a Non-Linux file system (like NTFS, FAT),--perms
causes errors because rsync cannot set Linux permissions on a non-linux file systems. Again, errors = no delete.-- When syncing to a Non-Linux partition, I use
--no-perms
to avoid those errors that sabotage--delete
,--delete-after
, or--delete-before
.If you still get errors after that, and cannot figure out how to make those errors go way, you can run a command that is exclusively dedicated to deleting the out-of-sync files:
The command above will delete stuff that's out of sync, but won't sync any files. So, you should sync again after this. That command is based off of this answer, except that I added the
--ignore-errors
argument also, so it would delete even if there are errors.The ‘-delete’ flag has to appear after your
/source
and/destination
.So it’s like this
The options are:
-r
recursive-v
verbose (print to screen)-p
permissions-t
time (use time instead of checksums ‘c’)-h
human readable-E
extended Info (the Color tags on a .Mac)