OpenSSH server on Ubuntu Impish fails to start after reboot, only manual restart works

Preface and issue

I have an Ubuntu 21.10 (Impish) server, which network is managed by netplan and systemd-networkd.

My /etc/netplan/01-netcfg.yaml looks like this:

network:
  version: 2
  renderer: networkd
  ethernets:
    eth0:
      addresses:
        - 1.2.3.4/32
        - dead:beef:f00:ba1::2/64
      routes:
        - on-link: true
          to: 0.0.0.0/0
          via: 1.2.1.2
        - on-link: true
          to: default
          via: fe80::1
  vlans:
    eth0.1234:
      id: 1234
      link: eth0
      mtu: 1400
      addresses:
        - 10.0.0.2/25
      routes:
        - on-link: true
          to: 10.0.0.128/24
          via: 10.0.0.1

ip a reports the correct IPs for the three interfaces (lo, eth0, eth0.1234@eth0). Ping and other network traffic works (in-/outbound, outside/vlan).

However, when I reboot the server, sshd.service fails to start at boot time with the following error:

Dec 14 19:18:02 srvnme sshd[1380]: error: Bind to port 22 on dead:beef:f00:ba1::2 failed: Cannot assign requested address.
Dec 14 19:18:02 srvnme sshd[1380]: error: Bind to port 22 on 1.2.3.4 failed: Cannot assign requested address.
Dec 14 19:18:02 srvnme sshd[1380]: fatal: Cannot bind any address.

While boot, the server does not show any issues related to the interface

Dec 14 19:18:02 srvnme kernel: [    1.240607] r8169 0000:22:00.0 eth0: RTL8168h/8111h, de:ad:be:ef:f0:00, XID 541, IRQ 154
Dec 14 19:18:02 srvnme kernel: [    1.240738] r8169 0000:22:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
Dec 14 19:18:02 srvnme kernel: [    5.046402] r8169 0000:22:00.0 eth0: Link is Down
Dec 14 19:18:03 srvnme kernel: [    8.054249] r8169 0000:22:00.0 eth0: Link is Up - 1Gbps/Full - flow control rx/tx
Dec 14 19:18:03 srvnme kernel: [    8.054273] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

As stated, after a server reboot, I have to hook up a KVM to the server to restart the OpenSSH server to be able to ssh into the server again.

Non-working solutions (so far)

I edited the OpenSSH server configuration (systemctl edit sshd) and tried two approaces:

[Unit]
Requires=systemd-networkd.service
After=systemd-networkd.service
BindsTo=systemd-networkd.service

[Unit]
Wants=network-online.target
After=network-online.target

Both approaches had no effect.

How to fix the OpenSSH does not start at server boot issue?