Home / server / Questions / 63809
Accepted
John Clayton
Asked: 2009-09-10 07:32:40 +0800 CST

ZFS: Mirror vs. RAID-Z

  • 772

I'm planning on building a file server using OpenSolaris and ZFS that will provide two primary services - be an iSCSI target for XenServer virtual machines & be a general home file server. The hardware I'm looking at includes 2x 4-port SATA controllers, 2x small boot drives (one on each controller), and 4x big drives for storage. This allows one free port per controller for upgrading the array down the road.

Where I'm a little confused is how to setup the storage drives. For performance, mirroring appears to be king. I'm having a hard time seeing what the benefit would be of using RAIDZ over mirroring would be. With this setup I can see two options - two mirrored pools in one stripe, or RAIDZ2. Both should protect against 2 drive failures, and/or one controller failure...the only benefit of RAIDZ2 would be that any 2 drives could fail. The storage should be 50% of capacity in both cases, but the first should have much better performance, right?

The other thing I'm trying to wrap my mind around is the benefit of mirrored arrays with more than two devices. For data integrity what, if any, would be the benefit of a RAIDZ over a three-way mirror? Since ZFS maintains file integrity what does RAIDZ bring to the table...doesn't ZFS's integrity checks negate the value of RAIDZ's parity?

opensolaris

3 Answers

  1. RAID-Z eliminates most of the write penalty and the data integrity issues that RAID 5/6 volumes suffer from, at the cost of some CPU time. Typically, systems have CPU cycles to spare, so spending CPU time to improve IO performance and data integrity is a good compromise vs. mirroring.

    Here is a detailed explanation of RAID-Z that may answer other questions.

    Also, remember that RAID is a fault-tolerance solution. You don't implement RAID-Z2 to protect against data loss -- you perform backups or replicate to do that. You choose to implement RAID-Z2 vs. RAID-Z or RAID-10 vs RAID-6 vs. RAID-5 to keep your systems operational in the event of hardware failure.

    • 18
  2. Best Answer

    The simple answer is that to mirror something takes almost no processing power - it just writes to the disk a second time. For RAID-Z2, you have to compute an entirely new parity block, which although small CAN bog down the CPU when you have to write large amounts of data quickly.

    Mirroring is always the preferred solution for high-speed data, if it's just bulk-storage without fast write speeds, RAID-Z2 is a good alternative that does allow any two drives to die as you allude to.

    The other advantage is that mirrored pools can be expanded with more mirrored devices - while a RAID-Z2 can not be expanded - though more RAID-Z2 storage can be added to the pool, it will be two RAID-Z2 storage pools concatenated (in effect) rather than equally split between all the storage and striped.

    • 15

  3. The main performance difference between mirrors and RAIDZ1/2 [1] is not the expected CPU usage (which we have plenty these days), but the fact that ZFS random IOPS depends on the total number of vdevs rather than the total number of disk.

    For example, 4x 2-way mirrors provides up to 8x the random read performance and 4x the random write performance of a single disk. On the other hand, a 8x RAIDZ2 vdev (6 data + 2 parity) is going to provide the same random IOPS of a single disk.

    In other words: for random IO heavy workload, mirrors is the best choice performance wise.

    [1]: RAIDZ3 is significantly heavier CPU wise.

    • 1