Our storage server has some problems, just ago we had a broken disk (WD 4TB RE SAS), though the raidcontroller (LSI MegaRAID 9271-8i) kept the disk online (status : ok). Only Media error showed 1 error. We decided to be save and replace the disk, during the resilvering a second and a tirth disk where marked as (resilvering) although only one disk showed 1 read error. Today the resilver is complete (no corruption, all oke), I started a scrub and was met with this :
zpool status
pool: data
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: scrub in progress since Thu Jan 14 10:50:00 2016
2.71T scanned out of 111T at 718M/s, 43h59m to go
0 repaired, 2.44% done
config:
NAME STATE READ WRITE CKSUM
data DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
br0c2 ONLINE 0 0 0
br1c2 ONLINE 0 0 0
br2c2 ONLINE 0 0 0
br0c3 ONLINE 0 0 0
br1c3 ONLINE 0 0 0
br2c3 ONLINE 0 0 0
r2c1 ONLINE 0 0 0
r1c2 ONLINE 0 0 0
r5c3 ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
7196084230607724634 FAULTED 0 0 0 was /dev/sdai1
r5c0 ONLINE 0 0 0
r0c1 ONLINE 0 0 0
r1c1 ONLINE 0 0 0
r3c1 ONLINE 0 0 0
r4c1 ONLINE 0 0 0
raidz2-1 ONLINE 0 0 0
r5c1 ONLINE 0 0 0
r0c2 ONLINE 0 0 0
r2c2 ONLINE 0 0 0
r3c2 ONLINE 0 0 0
r4c2 ONLINE 0 0 0
r5c2 ONLINE 0 0 0
r0c3 ONLINE 0 0 0
r1c3 ONLINE 0 0 0
r2c3 ONLINE 0 0 0
r3c3 ONLINE 0 0 0
r4c3 ONLINE 0 0 0
br0c0 ONLINE 0 0 0
br1c0 ONLINE 0 0 0
br2c0 ONLINE 0 0 0
br0c1 ONLINE 0 0 0
br1c1 ONLINE 0 0 0
br2c1 ONLINE 0 0 0
errors: No known data errors
/dev/sdai1 is online and the raidcontroller is not showing any error (not even media error). Can I try and put the disk online / offline ?
update : I try'd to deatach the disk but it refuses to do so, I was under to impression that raidz2-0 has 2 parity disks (and raidz2-1 also 2) so why can't I detach ?
zpool detach data 7196084230607724634
cannot detach 7196084230607724634: only applicable to mirror and replacing vdevs
Why are you passing the disks through a RAID Controller? JBOD would make more sense when using ZFS. You could run into Problems because of your controller.
Anyway, its save to just detach and re-attach the disk. You could also try to replace the disk (without really replacing it: zpool replace pool disk)
Let it resilver and scrub again.