I've been reading up on FreeNAS and RAID Z, and one of the things people kept mentioning is that you shouldn't use ZFS (or more specifically, RAID Z) on a machine without ECC RAM. I'm wondering what the reasoning behind this is. Is this an issue with RAID Z specifically? Is there any particularly heightened risk from not using ECC RAM with ZFS vs other file systems? Or is this just a general concern about corrupt memory spreading to a filesystem which would otherwise have corrected the error thanks to its support for self-healing redundant arrays?
You shouldn't use non-ECC RAM on any machine that is storing or processing data you care about. This is not something that is unique to ZFS.
You're right, ZFS devs have gone to great lengths to add many layers of hashing, verification, etc. to the filesystem - all of these are of massive help in ensuring data integrity, and all of these efforts can be undermined by a single undetected single bit flip in RAM.