Home / server / Questions / 543912
Accepted
84104
Asked: 2013-10-05 12:22:26 +0800 CST

Force update checksum on zfs?

  • 772

I recently changed the checksum property on one of my non-duplicated zfs filesystems to sha256 from on (fletcher4) to better support the sending of duplicated replication steams, as in this command zfs send -DR -I _starting-snaphot_ _ending-snapshot_.

However, the zfs manpage has this to say about send -D:

This flag can be used regardless of the dataset’s dedup property, but performance will be much better if the filesystem uses a dedup-capable checksum (eg. sha256).

The zfs manpage also states this about the checksum property:

Changing this property affects only newly-written data.

I have no desire to trust fletcher4. The tradeoff is that unlike SHA256, fletcher4 is not a pseudo-random hash function, and therefore cannot be trusted not to collide. It is therefore only suitable for dedup when combined with the 'verify' option, which detects and resolves hash collisions.

How can I update the filesystem's checksums, preferably without offlining the system?

zfs

1 Answers

  1. Best Answer

    To change the properties (be it compresson, deduplication or checksumming) of already written data, the zfs approach is to run the data through a zfs send | zfs receive sequence. Obviously, you do not need to offline the system for that, but you will need

    1. enough resources in your zpool / on the system to hold two dedup'ed copies of the data set in question
    2. downtime for the data set as you either would need to destroy it or rename it in the procedure
    3. enough time and patience for the operation to complete

    As you already are using deduplication for the zpool, running a zfs send | zfs receive with the destination on the same pool as the source would only use space needed for the newly-written metadata blocks. But be prepared for the copy to take a while - dedup can be awfully slow, especially if you do not have enough RAM to hold the entire dedup table in RAM.

    You obvisouly would need to cease all write operations to create the final, authoritative copy of the data set, but could minimize the downtime by copying off a snapshot first, stoping all writes and doing incremental zfs send -i | zfs receive as the final step.

    • 11