libvirt TCP tunneled network only works for 2 VMs

I'm setting up some VMs with libvirt (qemu driver). One VM is meant to be the gateway and is thus configured with two network interfaces. One interface is bridging into my real LAN and the other uses TCP tunneling (server mode):

<!-- External interface - i.e. facing the Internet -->
<interface type='direct' trustGuestRxFilters='no'>
  <source dev='eth0' mode='vepa'/>
</interface>

<!-- Virtual network -->
<interface type='server'>
  <source address='0.0.0.0' port='5558'/>
</interface>

Then I have two other VMs (called test_1 and test_2) which are practically identical (except for UUID, MAC and hostname of course). Those only have network interfaces that use the virtual TCP-tunneled network:

<interface type='client'>
  <source address='127.0.0.1' port='5558'/>
</interface>

Now the gateway VM should hand out IP addresses to all VMs on the virtual network - but it only does so to the test VM which is bootet up first. Also if both are running and I shut down the one which currently has an IP, the other one will get an IP.

I don't think it's a problem of the DHCP, because I can see that the second test VM doesn't even send DHCP request to the gateway. Also setting a static IP on the second VM doesn't help.

After all it just looks like there can only be one VM as a client to the virtual network and the other VM is not connected to the virtual network at all. But checking the connections on the host machine I can see that both qemu processes for the test VMs have TCP tunnels open to the host process of the gateway.

I'm really clueless. Any advice of what could be wrong or what I could try next?

EDIT: It's interesting to take a look at the actual connections on the host system:

qemu-syst 2900 libvirt-qemu   12u  IPv4  22574      0t0  TCP *:5558 (LISTEN)
qemu-syst 2900 libvirt-qemu   17u  IPv4  22639      0t0  TCP localhost:5558->localhost:38714 (ESTABLISHED)
qemu-syst 2946 libvirt-qemu   12u  IPv4  21820      0t0  TCP localhost:38714->localhost:5558 (ESTABLISHED)
qemu-syst 2967 libvirt-qemu   12u  IPv4  21835      0t0  TCP localhost:38717->localhost:5558 (ESTABLISHED)

Looks like the server (process 2900) only accepted the connection from the first VM (process 2949)