Home / server / Questions / 126851
Accepted
d-_-b
Asked: 2010-03-27 23:26:45 +0800 CST

linux LVM mirror vs. MD mirror

  • 772

I think I remember making some mirrors years ago with LVM, and I don't remember this "log" thing. Or maybe I made the mirror with mdadm and put LVM on top. That must be it.

  1. What is the LVM log for if it is just a mirror? What is stored there? What is it's purpose? Is using "--mirrorlog core" bad? What's the down side? I don't want to have to have another partition for logs if I don't have to.

  2. Any recommendations on using either technology? Even if I make the mirror with mdadm, I'll use LVM on top of that. So, in that case, maybe it's better to have the whole setup built with LVM...? Would that take more a performance hit or less?

The disks are for storing Xen domU "disks".

Sorry for the complex not-to-the-point "question".

Ideas and suggestions and links are most welcome. Thanks!

logging lvm mdadm mirror

5 Answers

  1. Best Answer

    Personally I always go MD+LVM. It is faster (MD can do parallel reads in RAID1) and it requires only 2 disks (if you do not want to rebuild the mirror after every reboot) and MD is designed just to do RAID, and it does it very well. It is also more flexible.

    Also I have been burned by LVM. In one case it was LVM striping on HP-UX, (should not happen w/ LVM2 in Linux), and I could not extend the LV because the extension had to have the exact same number of stripes. It seems reasonable, but sometimes ...

    Another time it was with mirroring (LVM2 in Linux this time), but now I cannot remember if LVM refused to take a snapshot of a mirrored LV or to extend it, effectively negating the benefit for having LVM in the first place (we did it in the end, but involved using lvconvert back and forth).

    • 10

  2. When a mirrored pair is abruptly shut down, in any system, there may be some blocks that have been written to one device but not yet to the other. Conceivably with reordering IO paths there are some blocks that have been written to A but not B, and some written to B but not yet to A.

    When the machine restarts, it needs to make the devices consistent. Just having the two devices alone is not enough: it wouldn't know which blocks are inconsistent and it wouldn't know which version was more recent.

    So the mirror software needs to store a bit of metadata to keep track of this. This is called the "write-intent log".

    md normally stores it in some reserved space inside the device itself, though you can also choose to store it on a separate device. (This is one reason why md mirrors are slightly smaller than the basis devices.

    LVM mirrors can only store the write-extent log externally, which as you note is a bit annoying for the typical case where you have exactly two physical disks. I believe you do still have the choice to store it on a different partition on the same physical device, but that's still inconvenient.

    The purpose of the LVM mirror log, as I understand it, is to keep track of blocks that have been written to one device but not yet to the other.

    • 5

  3. these are two different concepts:

    • LVM is as the name says it for volume management. Think of it as a mechanism to combine multiple volumes into one (yeah you can say it's like raid0), resizing them, live resizing, creating snapshots.
    • MD is for RAID management. Add multiple devices into one metadevice, make this metadevice highly available by using raid1, 5, 10, etc...

    LVM operates on storage devices be it metadevices, or devices presented by the raid controller, or just plain disks.

    MD operates on disks (yeah you can also use disks presented by the raid controller, but what's the point?..

    LVM is not for making your storage highly available. And likewise MD is not for volume management...

    Hope this helps :)

    • 2

  4. LVM came from other UNIX system types. I used it on HPUX which did not have the MD layer of Linux when I worked on it 10 years ago, not sure about now. Under those circumstances LVM mirroring becomes useful in a situation where hardware RAID is not available.

    • 1

  5. with the -mirrorlog mirrored and the -alloc anywhere option, with two disks, you get two 4M mirror logs, one on each disk. I'm annoyed that I have to override the policy for what seems like a most common situation, but oh well.

    LVM on Linux is not related to the thing on HP-UX or AIX, just has the same userspace commands. LVM on Linux internally uses the DM system, and therefore uses all the same code as MD. At this point, I see no advantage of LVM over MD. For partitions which you want fast and do not care about backups for (e.g. swap partitions for guest VMs), LVM gives you option of whether you want those mirrored or not, or whether stripping is better.

    And, it's way easier to add a disk to LVM than MD.

    • 1