Had to destroy and recreate a pool due to some other issues. I noticed that performance on the new pool is atrocious.
Whereas before on the old pool, ARC was full and ARC hits were high, ARC hits on the new pool have dropped considerably.
On the new pool, ARC just does not fill up.
The pattern of ARC hits are also strange. Having one or two clients try to play video, ARC would hit for a few seconds, and then plummet to 0, which causes the video playback to stutter and stop.
Why is performance so bad on the new pool?
Turns out that the new pool just needed to be scrubbed.
Without a scrub, it seems that the ZFS database was quite confused, perhaps still mapped to information from the old, deleted pool, and every new request to play a video clip would have it go looking for data on on the spinning disks.
After a scrub, ZFS could properly load files into the ARC, and good performance returned.