Home / server / Questions / 218105
In Process
Xiong Chiamiov
Asked: 2011-01-04 00:46:56 +0800 CST

Rebuilding LVM after RAID recovery

  • 772

I have 4 disks RAID-5ed to create md0, and another 4 disks RAID-5ed to create md1. These are then combined via LVM to create one partition.

There was a power outage while I was gone, and when I got back, it looked like one of the disks in md1 was out of sync - mdadm kept claiming that it only could find 3 of the 4 drives. The only thing I could do to get anything to happen was to use mdadm --create on those four disks, then let it rebuild the array. This seemed like a bad idea to me, but none of the stuff I had was critical (although it'd take a while to get it all back), and a thread somewhere claimed that this would fix things. If this trashed all of my data, then I suppose you can stop reading and just tell me that.

After waiting four hours for the array to rebuild, md1 looked fine (I guess), but the lvm was complaining about not being able to find a device with the correct UUID, presumably because md1 changed UUIDs. I used the pvcreate and vgcfgrestore commands as documented here. Attempting to run an lvchange -a y on it, however, gives me a resume ioctl failed message.

Is there any hope for me to recover my data, or have I completely mucked it up?

My LV was set up linearly. It seems that I should be able to recover at least the data from md0 by using something like TestDisk. However, examples show LVM partitions listed in TestDisk, while I only see sda,sdb,etc. and md0 and md1.

[$]> lvdisplay
  --- Logical volume ---
  LV Name                /dev/VolGroup/lvm
  VG Name                VolGroup
  LV UUID                b1h1VL-f11t-TGJv-ajbx-4Axn-wi89-fReUKu
  LV Write Access        read/write
  LV Status              suspended
  # open                 0
  LV Size                6.14 TiB
  Current LE             1609648
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:0

[$]> lvchange -ay /dev/VolGroup/lvm
  device-mapper: reload ioctl failed: Invalid argument
linux lvm raid mdadm

2 Answers

  1. I'm afraid that, by combining two (RAID) disks into a LVM volume and then deleting and recreating one of those disks, you actually have lost your data. If you used LVM striping, they're definitely gone, since every other chunk of your files was on that disk. With linear mode, you may be able to regain a part of your files with some disk recovery software.

    Pity, because your basic configuration is sensible. Because of RAID5, you should have been able to access the volume even with one disk missing. I'd have suggested a backup before the "mdadm --create"...

    • 4

  2. Next time, you've two options ... either you can manually attempt to assemble the raid using mdadm --assemble instead of --create. Or you can use "ckraid" similarly to the filesystem check, but for raid 5 arrays. ( The Linux Software Raid How-To mentions the latter ).

    Personally, though, if you're using SW Raid without a UPS I'd personally go for Raid 10 instead of 5 or 6, since it should be more tolerant of power issues.

    • 1