Home / server / Questions / 1039318
Accepted
ascendants
Asked: 2020-10-20 09:31:59 +0800 CST

ZFS - is it safe to remove files served by ZFS during a resilver/rebuild?

  • 772

I have a Nexenta ZFS system serving a large NFS volume (using ~85% of 250 TB). One of the 70 disks failed over a week ago, and the system is resilvering a hot spare without issues (other than the large performance losses due to the intense resilvering I/O).

I know that having more free space will reduce the time needed for future resilverings, so I'm planning to clean up ~70 TB from the NFS volume soon.

But, I'm not sure if doing the cleanup during the current resilver will cause problems, or if it'd reduce the time needed for the current resilver, which brings up my question:

  • will deleting files negatively impact an ongoing ZFS resilver?

System info:

# uname -a
SunOS stor-nas02a 5.11 NexentaOS_4:55745843a2 i86pc i386 i86pc

Array info:

  • 7x raidz2 (10 disks each)
  • 2x SLOG mirrors
  • various spares and caches
  • status:
action: Wait for the resilver to complete.
  scan: resilver in progress since _________
    29.7T scanned out of 199T at 60.2M/s, (scan is slow, no estimated time)    # the 199 TB is compressed
    451G resilvered, 14.92% done
zfs opensolaris raidz sunos nexenta

1 Answers

  1. Best Answer

    Any userland storage operations are fine to perform, but they do effectively halt the resilvering while they're being performed. ZFS prioritizes normal operations (using the file system) over resilvering operations. Resilvering reads all the data in the pool in a nonsequential order and writes only the required data to the new disk(s). Deletes aren't much different than writes, especially since I imagine you probably have snapshots which mean that the deletes are basically non-operations in terms of disk hits.

    Destroying snapshots during a resilver is something I wouldn't recommend. But it's not out of safety or a concern for data loss, it's because it makes the pool useless. A several year old memory tells me that performance hit on a small (25TB 12x2TB, ZIL and SLOG on ~fast SSDs) pool destroying ~1TB on a single disk resilver made the system unusable for around a day.

    In short, ZFS does everything it can to provide you with reliable access to your data, and let you use it normally, while it does all its housekeeping in the background. Resilvers are critical, but they're also interruptible background tasks.

    More info about how resilvering works can be found here, and in the Oracle docs.

    • 3