julianjm's questions

I'm reinstalling a workstation (not a headless server) with an Intel RAID card with BBU and 4GB internal cache.

My current setup uses ZFS on a pair of mechanical disks and a partition on an SSD for the ZIL.

I want to keep using ZFS on 2 bigger disks, mainly because storage flexibility: filesystems and block devices. That comes handy for lxc containers, ability to make regular snapshots, etc.

The controller i'm about to use doesn't support write-caching on JBOD drives, but it does on 1 disk volumes with RAID0. That's the closest it gets to JBOD without loosing the battery backed write cache.

The main concern I have is the lack of SMART information available to ZFS. I can query the drives directly with smartctl -d megaraid,S /dev/sdb, being S the drive slot on the controller. I'm not sure if ZFS if capable of using that.

As I said, this is for a personal workstation. What do you think? Having thousands of fsyncs/s thanks to the write cache, do I need ZIL o can I live without it?

Any specific settings on the controller (regarding the SMART management) that I should take into account?

Thanks

I've been reading lately about write caching, NCQ, firmware bugs, barriers, etc regarding SATA drives, and I'm not sure what's the best setting that would make my data safe in case of a power failure.

From what I understand, NCQ allows the drive to reorder the writes to optimize performance, while keeping the kernel informed on which requests have been physically written.

Write cache makes the drive serve a request much faster, because it doesn't wait for the data to be written to physical disk.

I'm not sure how NCQ and Write cache mix here...

Filesystems, specially journalled ones, need to be sure when a particular request has been written down. Also, user space process use fsync() to force the flush of a particular file. That call to fsync() shouldn't return until the filesystem is sure that the data is written to disk.

There's a feature (FUA, Force Unit Access), which I've seen only on SAS drives, which forces the drive to bypass cache and write directly to disk. For everything else, there's write barriers, which is a mechanism provided by the kernel that can trigger a cache flush on the drive. This forces all the cache to be written down, not just the critical data, thus slowing the whole system if abused, with fsync() for example.

An then there are drives with firmware bugs, or that deliberately lie about when data has been physically written.

Having said this.. there are several ways to setup the drives/filesystems: A) NCQ and Write cache disabled B) Just NCQ enabled C) Just Write cache enabled D) Both NCQ and write cache enabled

I'm asuming barriers are enabled.. BTW, how to check if they are actually enabled?

In case of power loss, while actively writing to the disk, my guess is that option B (NCQ, no cache) is safe, both for filesystem journal and data. There may be a performance penalty.

Option D (NCQ+cache), if using barriers or FUA, would be safe for the filesystem journal and applications that use fsync(). It would be bad for the data that was waiting in the cache, and it's up to the filesystem to detect it (checksuming), and at least the filesystem won't be (hopefully) in an unstable state. Performance-wise, it should be better.

My question, however, stands... Am I missing anything? Is there any other variable to take into account? Is there any tool that could confirm this, and that my drives behave as they should?