Home / server / Questions / 1087737
In Process
Karl
Asked: 2021-12-20 10:01:06 +0800 CST

How to pass a cloud config to LXD?

  • 772

I've been to countless sites including this one, trying to figure out how to make LXD run a cloud-config when I launch my LXD container. Some places recommend setting up a profile (didn't work). Others recommend redirecting a YAML file into the lxc command (didn't work), and others recommend using the --config option and passing the file in that way (didn't work). Some places say I have to add #cloud-config to my config, others don't bother. Some even recommend using an XML file. I'm obviously missing some critical piece of information that everyone else just does by default, but I can't figure out what it is.

My updated "simple example" that tries to install tree and tries to touch /run/cloud-config-did-run:

lxc delete -f x

cat << EOF >config.yml
#cloud-config
output: {all: '| tee -a /var/log/my-cloud-init-output.log'}
package_update: true
package_upgrade: true
package_reboot_if_required: true
packages:
  - tree
runcmd:
  - touch /run/cloud-config-did-run
EOF

lxc launch ubuntu: x --config=user.user-data="$(cat config.yml)"
sleep 5
lxc exec x -- bash -c "ls /run"
lxc exec x -- bash -c "tree /etc"

The output DOES get directed to /var/log/my-cloud-init-output.log, so it is getting processed, but nothing other than the output directive runs (the logs don't even mention any other things running, or any errors - just the standard SSH keygen stuff).

Maybe the indentation is wrong? Or the config is in the wrong subtree? Or there's some magic value missing? A version of LXD where this is broken? (I'm running version 4.20). I've been at this for 10 hours so far and no matter what I do, my cloud config is completely ignored (no errors, no logs, no record of anything running, no record that I ever instructed it to do anything - other than the standard ssh keygen stuff that's apparently baked in). Can someone please turn the above into a working example that is guaranteed to run if I just paste it into a shell?

lxd cloud-config

1 Answers

  1. The available (non-)information about cloud-config is really a pain. And the docs are still bad. But first things first:
    I tried exactly your code and succeeded (package upgrades + /run/cloud-config-did-run worked). Tested with Ubuntu 20.04 as well as 22.04.

    BUT, I work on LXD 5.1. So this might be the important key difference.

    Some remarks which might be useful:

    • I configure a simple output to differentiate between the stages init, config, final and therein between standard log and errors. That makes debugging easier.

      output:
  init: ["> /tmp/cloud-init.log", "> /tmp/cloud-init.err"]
  config: [ "> /tmp/cloud-init-config.log", "> /tmp/cloud-init-config.err" ]
  final: [ "> /tmp/cloud-init-final.log", "> /tmp/cloud-init-final.err" ]

    • As soon as I have fired the lxc launch command
      I change into the container with lxc shell x.
      With cloud-init status --wait I look if everything works ("done") or fails.
      If it fails /tmp/*.err is your friend

    • Until now I never had an issue with writing to /tmp. Cloud-init changed the order of execution somewhere in the past. That might have solved it.

    • useless knowledge: package_update is included in package_upgrade

    • #cloud-config is needed. I tried without and it did not work. The YAML was ignored altogether

    • About the cloud-config format: it is YAML (the file extension does not matter). I don't think anything else works for LXD - besides a MIME multi-part archive

    • MIME multi-part archives work with LXD and its a good way to reuse code for different configs. Its easy to use but the docs seem rudimentary.
      A little example with your config.yml together with a shell script:

      cat << EOF >test.sh
#!/usr/bin/env bash
touch /run/here-we-go-with-a-shellscript
EOF

cloud-init devel make-mime -a config.yml:cloud-config -a test.sh:x-shellscript-per-once > newconfig
lxc launch ubuntu:j f --config=user.user-data="$(cat /root/newconfig)"
lxc shell f
cloud-init status --wait
ls /run

      A good starting point for working with MIME multi-part archives is probably this cloudinit doc page

    • LXD Profiles with cloud-config worked when we tested it, but we don't use it

    I hope that helps somehow. If you are interested I could add a larger cloud-init yaml which we tested for setting up a typical web container behind a reverse proxy.

    • 0