Home / ubuntu / Questions / 1323004
Accepted
i9pp0
Asked: 2021-03-13 04:24:21 +0800 CST

How to get rsync to synchronize only new blocks for blockchain

  • 772

Every time I run rsync to backup a blockchain, it starts from the beginning and tries to sync the whole thing, every time, even though there may only be a few blocks worth of data that is new. The rsync man page says, "It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination," but I cannot figure out how to get it to only update the new info that has been added to the blockchain. As the chain is very large, it is ridiculous to keep copying the whole thing every day. Can rsync backup only new data, and if so, how can I tell it to do so?

This what I am using:

> rsync -avz --exclude-from=/exclude/file -e ssh /from/file [email protected]:/backup/file
backup

2 Answers

  1. Best Answer

    According to this article, a combination of the following properties should work for you:

    --partial --inplace --append

    In your case that would be:

    rsync -avz --partial --inplace --append --exclude-from=/exclude/file -e ssh /from/file [email protected]:/backup/file

    It also makes sense to me that a blockchain would only need "appended" data, i.e. added blocks.

    In addition, --progress can be added as well.

    • 1

  2. I believe you need the --inplace option.

    Normally, rsync creates a new file for every transfer and removes the old one when the transfer was successful, so any interrupt will still keep one version of the file intact. With --inplace it writes the existing file directly, so an interrupt may leave an inconsistent file behind; that's why it's not the default.

    • 0