Home / server / Questions / 1166195
Accepted
Igor Popov
Asked: 2024-10-05 23:50:35 +0800 CST

Is it safe to use --append option of rsync to back-up Outlook pst and ost files and Thunderbird files

  • 772

I plan to backup Outlook emails from desktop computers to NAS. The tool of choice is rsnapshot as I can install it on the old NAS (Synology DS 1010+). Rsnapshot uses rsync for its job. The classic Outlook (2016) saves messages in pst or ost files, which can grow to gigabytes when contain many email messages. The --append rsync option can considerably speed up the backup process of large files. I am concerned if it is safe to backup pst and ost files using this option. Namely, after receiving or sending a new email Outlook probably appends these emails to corresponding pst or ost files. However, when one deletes an email, how is it managed in pst and ost files, i.e. is a part in the mid of a pst/ost file removed? If so, this would make the --append option not-applicable. So, my question states, is it safe to use the --append option for backups of Outlook pst and ost files.

Related to this, Thunderbird saves emails to files (without extension) and besides them it writes some msf files which are much smaller than the former ones. Is it safe to use --append rsync option to backup the large files without extension (which contain actual emails)?

rsync

2 Answers

  1. Best Answer

    No, and no.

    PST and OST files, which are internally the same format, are indexed databases. Appending the new stuff on the end doesn't correctly update indices, and Outlook will also re-index and compact PST and OST files to save space when it finds sufficient messages have been deleted, or when it finds mild mailbox corruption, or when it feels like it.

    By the same token, Thunderbird's msf files are indexes into the mailbox files, and while appending normally will work, Thunderbird will also compact and reindex files when it determines that sufficient space (in my case, 200MB) can be recovered by that process. That compaction step would invalidate the append mechanism.

    • 2

  2. Using the --append option with rsync for backing up Outlook PST and OST files can be problematic. Here’s why:

    • PST and OST File Structure: Outlook PST and OST files are not simple append-only files. When emails are deleted or moved, the file structure can change significantly, including the removal of data from the middle of the file. This means that using --append could result in incomplete or corrupted backups because it assumes that new data is only added to the end of the file. File Integrity: The --append option is designed for files that grow by appending data at the end, like log files. Since PST and OST files can have data removed or modified throughout the file, --append is not suitable for these types of files.

    For Thunderbird, the situation is similar:

    • Email Storage Files: Thunderbird stores emails in MBOX format (files without extension) and uses MSF files for indexing. While the MBOX files are generally append-only, deletions and other modifications can occur, making the --append option risky. File Consistency: Using --append with Thunderbird’s MBOX files could lead to inconsistencies if emails are deleted or moved, as the file structure can change.

    Recommended Approach

    Instead of using --append, consider the following options:

    • Regular Full Backups: Perform regular full backups of the PST, OST, and MBOX files. This ensures that you have a complete and consistent copy of your data.
    • Incremental Backups: Use rsync without the --append option to perform incremental backups. This way, only the changes are transferred, but the entire file is checked for modifications.
    • Snapshot Backups: Since you’re using rsnapshot, it already handles incremental backups efficiently by creating snapshots. Ensure that rsnapshot is configured correctly to capture the entire file each time.
    • 0