Home / server / Questions / 189380
In Process
Vincent
Asked: 2010-10-10 11:19:28 +0800 CST

Using 4k drives with md/LVM/dm-crypt

  • 772

I know that I have to align my 4k drives by a multiple of 8 sectors, but what about md-RAID / LVM / dm-crypt? How do I tell these layers that my drive is 4k? If they don't respect the 4k sector size, the partition alignment is useless. How do I align LVM/md/crypto-layers? Thanks.

linux lvm mdadm advanced-format

4 Answers

  1. Be careful! gpt labels, required for disks > 2 TiB, are 39 (512-byte) sectors long. So if you create your first partition immediately after the label, it won't be on a 4KiB boundary.

    GNU parted does not do this by default, probably because many "Advanced Format" drives falsely claim that their physical sectors, not just their logical sectors, are still only 512B.

    So if you're using GNU parted, ensure that each partition starts on an LBA divisible by 8 (LBAs remain 512B, so 8*512B = 4KiB). LBAs originate at 0, so start the first partition at "40s".

    Also, if you use GRUB, leave room for its second stage bootstrap. MS-DOS labels are 63 sectors, with enough unused room for GRUB to stash its second stage bootstrap, but there's no unused space in a gpt label. So make a small partition 1, set its "bios_grub" flag, and then create your "real" partitions after that -- making sure that each and every one begins on a LBA that's a multiple of 8.

    • 4

  2. See https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues

    The short version is that if you have a recent distro, it should automatically do the right thing. For older distros, it's a bit more complicated.

    For LVM you should investigate the --dataalignment option to pvcreate, or for even older distros -–metadatasize.

    MD, AFAIK, puts its own metadata at the end of the partitions, so it should always be aligned to the underlying partition.

    For mkfs, again the filesystem should be aligned with the underlying partition. For some filesystems you can add options for stripe width and stripe size in case you're running on a RAID device, so that the filesystem can try to align stuff on RAID stripe boundaries.

    • 3

  3. problem is mostly with alignment of partition beginning with structure of underlying disk. to keep backwards compatibility disks 'lie' to the bios/os that they have 512B sectors, while in fact they have 4096B sectors in case of modern hard drives, 32-64kB sectors in case of most common stripping raids/ssds.

    misaligned partitions will hurt your performance. i have done some benchmarks only on regular partitions on the top of the disk - without lvm and my results measured with bonnie++ were without proper alignment:

    Sequential Output Block: 29MB/s
Sequential Output Rewrite: 20MB/s

    with alignment:

    Sequential Output Block: 70MB/s
Sequential Output Rewrite: 37MB/s

    check this for lvm alignment.

    • 1

  4. Most newer distributions are updated to know about the 4K thing by now. I just built a md-RAID/LVM/XFS setup on a bunch of 2TB drives with no problems. Didn't do anything special.

    • 0