Questions[containers](server)

Debian 12.1 (6.1.0-11-amd64) running LXD/LXC and on an unprivileged container setting security.idmap.isolated=true seems to fail to update the owner/group of the container's files.

Here is an example:

# lxc launch images:debian/12 debian
(...)

# lxc config get debian volatile.idmap.base
296608

# lxc stop debian
Error: The instance is already stopped

# lxc config set debian security.idmap.isolated true

# lxc config get debian security.idmap.isolated
true

# lxc start debian

Now if I list the files on the container volume I'll get they're all owned by the host root user:

# ls -la /mnt/NVME1/lxd/containers/debian/rootfs/
total 24
drwxr-xr-x 1 root   root  154 Sep  5 06:28 .
d--x------ 1 296608 root   78 Sep  5 15:59 ..
lrwxrwxrwx 1 root   root    7 Sep  5 06:25 bin -> usr/bin
drwxr-xr-x 1 root   root    0 Jul 14 17:00 boot
drwxr-xr-x 1 root   root    0 Sep  5 06:28 dev
drwxr-xr-x 1 root   root 1570 Sep  5 06:28 etc

I tried multiple versions of LXD/LXC. This happens with both 5.0.2 from apt as well with 4.0 and 5.17 (latest) from snap.

Interestingly enough I have another Debian 10 (4.19.0-25-amd64) running and older LXD 4 from snap and on that one things work as expected:

# ls -la /mnt/NVME1/lxd/containers/debian/rootfs/
total 0
drwxr-xr-x 1 1065536 1065536  138 Oct 29  2020 .
d--x------ 1 1065536 root      78 Oct 14  2020 ..
drwxr-xr-x 1 1065536 1065536 1328 Jul 24 19:07 bin
drwxr-xr-x 1 1065536 1065536    0 Sep 19  2020 boot
drwxr-xr-x 1 1065536 1065536    0 Oct 14  2020 dev
drwxr-xr-x 1 1065536 1065536 1716 Jul 24 19:08 etc

As you can see on this systems all the files are owned by 1065536:1065536.

Update:

I tried to probe around the maps with lxc config show debian in both machines and I saw this:

Machine running Debian 10:

security.idmap.isolated: "true"
(...)
volatile.idmap.base: "1065536"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1065536,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":1065536,"Nsid":0,"Maprange":65536}]'

Machine running Debian 12:

security.idmap.isolated: "true"
(...)
volatile.idmap.base: "231072"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":231072,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":231072,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":231072,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":231072,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[]'

Update:

I also tried a fresh install of Debian 11 (5.10.0-25-amd64) and it works as expected:

root@vm-debian-11-cli:~# ls -la /mnt/NVME1/lxd/containers/debian/rootfs/
total 24
drwxr-xr-x 1 1065536 1065536  154 Sep  6 06:28 .
d--x------ 1 1065536 root      78 Sep  6 15:31 ..
lrwxrwxrwx 1 1065536 1065536    7 Sep  6 06:25 bin -> usr/bin
drwxr-xr-x 1 1065536 1065536    0 Jul 14 17:00 boot
drwxr-xr-x 1 1065536 1065536    0 Sep  6 06:28 dev
drwxr-xr-x 1 1065536 1065536 1570 Sep  6 06:28 etc

Why didn't it populate volatile.last_state.idmap: '[]'? As with works with both Debian 10 and 11 apparently this can be related to the new kernel and/or its configuration.

How can I fix it? Thank you.

I wonder how security experts think about making container traffic secure. Let's take a simple K8S cluster as an example.

I guess we all agree that running HTTPS instead of HTTP within each container is more secure. I would normally configure a TLS on Ingress and then route to an internal container will TLS configured as well.

Modern mashing solutions usually install proxy sidecar - let's take Linkerd2 for a concrete example. Ingress will upgrade all incoming traffic to TLS, it will then use Linkerd's TLS to forward traffic to the sidecar proxy. Everything transmitted until here is TLS and secured. I wonder, now that a secured traffic "already reached" the pod, should a container use TLS as well, or is it safe enough to communicate with the sidecar without TLS? I'd like to hear the security aspect between the container and its sidecar. Should a sidecar be treated as a stand-alone container and thus the communication with a container should also be TLS secured?

In AWS for example, when I spin up a new EC2 instance, it loads up a new VM, then populate the VM with a container image. This is the reason why spinning up new EC2 instances take 60-90 seconds to start.

Out of curiosity, what are the disadvantages to having AWS run the host machine as-is, and when a user wants to "spin up an EC2 instance", AWS just spins up a container with restricted permissions, and allows the user only access to that container?

The upside would be that the compute instance would spin up very quickly. I'm still learning about cloud technologies, so I was just wondering what the downsides are.

Perhaps it is harder to allocate CPU resources without using VMs? And as a result, users would fight over each other to take the available CPU? Or perhaps there's some security concern? Would love to learn about this.

we are running some jails in FreeBSD 10 and the decision has been made to decommission the FreeBSD server and migrate all the jails to Linux environment Red Hat 6 or 7.

Is there a straight forward way of migrating BSD jails to linux -- as containers that can be run via docker

or any other way to convert BSD jails to containers and pull/run via docker again.

Any other way of doing it ? I'm unsure whether this is actually possible or not because I tried reading on google but has not come across anything like that.

I am no LXC expert, but as far as I know it is really similar to linux-vserver.

If that is correct, I wonder why there is another player in the already crowded virtualization camp? What does LXC provide (or promise) that linux-vserver doesn't have?

I'm a long time linux-vserver user, I remember it was around many years ago before all this virtualization hype. Development slowed down in recent times, but never totally stopped - I know because developers helped me to bring it back to a useful state on linux/hppa (in Gentoo, at least). They regularly push out patches against the latest kernels, using all the latest goodies (the same container infrastructure used by LXC) and even integrating with GRsecurity.

People who started the LXC effort probably had a good reason to do so instead of joining the already well established linux-vserver base? Then I'd like to know.

Is it all about mainline integration? Then why not adapt linux-vserver for integration, like Xen people are doing with XenLinux?