Home / server / Questions / 840167
In Process
Simone Fantini
Asked: 2017-03-24 06:03:28 +0800 CST

LXD migrate containers to new storage backed

  • 772

On my host, I have Ubuntu 16.10:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.10
Release:    16.10
Codename:   yakkety

and I use LXD from the stable repo:

lxc --version
2.12

I currently have a few containers created with DIR as the storage backend:

root@Ubuntu-1610-yakkety-64-minimal ~ # lxc storage show default
config:
  source: /var/lib/lxd/storage-pools/default
name: default
driver: dir
used_by:
- /1.0/containers/elasticsearch-internal
- /1.0/containers/yyy-dev
- /1.0/containers/yyy-pre-prod
- /1.0/containers/xxx-dev
- /1.0/containers/xxx-dev/snapshots/snap1
- /1.0/containers/mysql-dev
- /1.0/containers/mysql-dev/snapshots/snap01
- /1.0/containers/mysql-preprod
- /1.0/images/2cab90c0c342346ea154bc2e8cacdae752a70747a755ce1f2970c9a9ebb5fe8c
- /1.0/images/d51e7b34d5f470912bc45a6270278d7990b049d826e33dd8affe9b54aaf0d7ee
- /1.0/profiles/default

I also have 2 x 2TB SATA III HDDs mirrored with ZFS:

root@Ubuntu-1610-yakkety-64-minimal ~ # zpool list -v
NAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
lxdstorage  1.81T   361M  1.81T         -     0%     0%  1.00x  ONLINE  -
  mirror  1.81T   361M  1.81T         -     0%     0%
    sdc      -      -      -         -      -      -
    sdd      -      -      -         -      -      -

Now, since version 2.12, LXD has the new storage API: lxd/storage-backend

I need to move the actual containers from DIR to ZFS storage. What I have done so far is create the new storage:

lxc storage create pool1 zfs source=lxdstorage/containers

root@Ubuntu-1610-yakkety-64-minimal ~ # lxc storage list
+---------+--------+------------------------------------+---------+
|  NAME   | DRIVER |               SOURCE               | USED BY |
+---------+--------+------------------------------------+---------+
| default | dir    | /var/lib/lxd/storage-pools/default | 11      |
+---------+--------+------------------------------------+---------+
| pool1   | zfs    | lxdstorage/containers              | 0       |
+---------+--------+------------------------------------+---------+

How can I move the containers from default to pool1?

zfs

2 Answers

  1. There is still no direct way to do this (today version 2.14).

    The workarround is stop container, publish as image, delete original container and init it in new storage pool:

    lcx stop c1
lxc lxc publish -f c1 --alias c1
lxc delete c1
lxc init c1 c1 -s <new pool>
lxc start c1
lxc image delete c1
    • 2

  2. LXC Manual migration from dir: to zfs: backend on Debian Stretch

    I could not find a way on SE which worked for me, i do not seem to have the publish option in debian lxc. 

    dpkg -l lxc 
#  1:3.1.0+really3.0.3-8

    My containers live in ext4 on /var/lib/lxc (default install), and I need them on zfs in /tank/lxc/containers

    0) Stop the container(s):

    lxc-stop mycontainer

    1) Create a file /etc/lxc/lxc.conf - lxc will pick this up and works from zfs. If you forgot to stop a container (like i did), move the file, then stop the container, and put back this file again

    # /etc/lxc/lxc.conf
lxc.lxcpath = /tank/lxc/containers
lxc.bdev.zfs.root = tank/lxc/containers

    2) Create the datasets:

    zfs create tank/lxc
zfs create tank/lxc/containers

    3) Create the new container with the same name (this will land on zfs now)

    lxc-create mycontainer

    Turns out mycontainer is a ZFS dataset which is not mounted. You could start it first to initialize it, or you can copy the old config over too (mac address, autostart, and what else you had set).

    4) The 2nd dataset you mount here is in fact the rootfs of the container(!)

    zfs mount tank/lxc/containers
zfs mount tank/lxc/containers/mycontainer
# this is attaching on /tank/lxc/containers/mycontainer/rootfs

    5) sync the old root to the new one

    rsync -av --delete /var/lib/lxc/mycontainer/rootfs/ /tank/lxc/containers/mycontainer/rootfs/

    6) copy the config if you need it

    cp /var/lib/lxc/mycontainer/config /tank/lxc/containers/mycontainer/config

    7) umount the datasets and start up your container from ZFS

    zfs umount tank/lxc/containers/mycontainer
zfs umount tank/lxc/containers
lxc-start mycontainer

    Repeat for all - Happy containering on ZFS!

    • 1