Home / server / Questions / 849088
Accepted
eckes
Asked: 2017-05-10 08:37:38 +0800 CST

Automatically distribute lvm (stripes) for specific LVM2 VG on Linux

  • 772

With Linux lvm2 I can create logical volumes inside a VG. When I specify with the lvcreate command striping/stripesize I can manually distribute the logical volumes over all physical volumes (as shown for lvnova below).

Is there a way to configure that lvcreate will always stripe and especially that if I create new LVs that they should be created round robin on the devices? (i.e. without specifying a physical "start" device in each lvcreate)

I need this to be configured persistently for a volume group which is used by OpenStack Nova for ephemeral devices. It looks like I cannot configure the lvm driver to actually distribute them more evenly.

I was looking for a way to specify it as attributes on the VG (since lvcreate is talking about inheriting allocations), but I cant find it in the manual.

The LVs have allocation policy inherit and the VG tells me it is normal:

# vgchange vgdata --alloc normal
Volume group allocation policy is already normal

I was also looking if there is a allocation policy inside lvm.conf. But there are only parameters for the cling2 strategy which does especially keep segments close together when extending a volume (which is not my main concern here).

Sample for a manually striped LV and for a OpenStack generated one volumes on a 4 physical volume system:

nvme0n1                                                               259:0    0   1.5T  0 disk
└─nvme0n1p1                                                           259:4    0   1.5T  0 part
  ├─vgdata-lvnova (dm-0)                                              252:0    0   750G  0 lvm  /srv/data
  ├─vgdata-ec3d1f0d--76f2--4cb2--af66--665dba0582b1_disk (dm-3)       252:3    0    50G  0 lvm
  ├─vgdata-ec3d1f0d--76f2--4cb2--af66--665dba0582b1_disk.swap (dm-4)  252:4    0     8G  0 lvm
  ├─vgdata-290ef35d--6d84--49cf--8d7d--3240dcd84a3e_disk (dm-5)       252:5    0    20G  0 lvm
 ...
nvme1n1                                                               259:1    0   1.5T  0 disk
└─nvme1n1p1                                                           259:5    0   1.5T  0 part
  └─vgdata-lvnova (dm-0)                                              252:0    0   750G  0 lvm  /srv/data
nvme2n1                                                               259:2    0   1.5T  0 disk
└─nvme2n1p1                                                           259:6    0   1.5T  0 part
  └─vgdata-lvnova (dm-0)                                              252:0    0   750G  0 lvm  /srv/data
nvme3n1                                                               259:3    0   1.5T  0 disk
└─nvme3n1p1                                                           259:7    0   1.5T  0 part
  └─vgdata-lvnova (dm-0)                                              252:0    0   750G  0 lvm  /srv/data

root@compute18:~# lvs --segments
  LV                                             VG     Attr      #Str Type    SSize
  lv-root                                        vg0    -wi-ao---    1 linear  223.52g
  lv-swap                                        vg0    -wi-ao---    1 linear   46.56g
  17bad6e1-41f0-4725-a74d-7a1f1df5f0b6_disk      vgdata -wi-ao---    1 linear   20.00g
  17bad6e1-41f0-4725-a74d-7a1f1df5f0b6_disk.swap vgdata -wi-ao---    1 linear    1.00g
  189585e0-9b69-4d7c-b1b4-194156312553_disk      vgdata -wi-ao---    1 linear   20.00g
...
  lvnova                                         vgdata -wi-ao---    4 striped 750.00g

BTW: I used a striped /dev/md0 before but I was hoping to get rid of this indirection.

This is Linux 4.4.0 on Ubuntu Trusty (lvm 2.0.2.98(2)) with KVM/nova OS Liberty.

linux

2 Answers

  1. You can configure this in /etc/lvm.conf. If you set raid_stripe_all_devices=1 the default should be striped instead of linear. You should also take a look at use_linear_target.

    • 3
  2. Best Answer

    The allocation/raid_stripe_all_devices=1 option in lvm.config mentioned by Andreas is only available with Ubuntu Zesty, and even then it seems to not turn on striped volumes by default.

    For this reason we patched OS:

    /usr/lib/python2.7/dist-packages/nova/virt/libvirt/storage/lvm.py
  else:
    check_size(vg, lv, size)
-   cmd = ('lvcreate', '-L', '%db' % size, '-n', lv, vg)
+   cmd = ('lvcreate', '-i', '4','-L', '%db' % size, '-n', lv, vg)

    This does it for our case:

    nvme0n1                                                              259:0    0   1.5T  0 disk
└─nvme0n1p1                                                          259:4    0   1.5T  0 part
  ├─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk (dm-5)      252:5    0   100G  0 lvm
  └─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk.swap (dm-6) 252:6    0     8G  0 lvm
nvme1n1                                                              259:1    0   1.5T  0 disk
└─nvme1n1p1                                                          259:5    0   1.5T  0 part
  ├─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk (dm-5)      252:5    0   100G  0 lvm
  └─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk.swap (dm-6) 252:6    0     8G  0 lvm
nvme2n1                                                              259:2    0   1.5T  0 disk
└─nvme2n1p1                                                          259:6    0   750G  0 lvm  /srv/data
  ├─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk (dm-5)      252:5    0   100G  0 lvm
  └─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk.swap (dm-6) 252:6    0     8G  0 lvm
nvme3n1                                                              259:3    0   1.5T  0 disk
└─nvme3n1p1                                                          259:7    0   1.5T  0 part
  ├─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk (dm-5)      252:5    0   100G  0 lvm
  └─vgdata-b063d1b9--e1ac--4cab--98d8--9a2d1c71406e_disk.swap (dm-6) 252:6    0     8G  0 lvm

# lvs --segments
  b063d1b9-e1ac-4cab-98d8-9a2d1c71406e_disk      vgdata -wi-ao---    4 striped 100.00g
  b063d1b9-e1ac-4cab-98d8-9a2d1c71406e_disk.swap vgdata -wi-ao---    4 striped   8.00g

    We will open an OpenStack bug for this missing setting.

    • 0