ping packet loss in wireguard UDP over TCP + UDP over socat tunnels

I have a wireguard tunnel mostly working between two Debian systems, but I had to workaround a ISP firewall (firewall X in the diagram below) that blocks everything inbound and prevents Server B from listenting for wireguard UDP packets directly.

In order to workaround the ISP firewall, I used a server of mine (Server A) on a public static IP address. Server B is protected behind the ISP firewall X, which allows only outbound connections. Server B does not have any public static IP addresses either, so it initiates a SSH connection to Server A in order to establish the reverse forwarding TCP tunnel from A to B. Server A then needs to act as simulated UDP endpoint for the wireguard connection, but wireguard is not installed there. The UDP packets need to travel from Server A to Server B using the SSH reverse forwarding tunnel established beforehand by Server B.

Implementation wise, I keep socat running on Server A and listening on 0.0.0.0:12345/udp. The idea is to make it forward packets to the real wireguard server (on Server B) wrapping the UDP packets it receives into the SSH TCP reverse forwarding tunnel.

That said, Server B is actually a bare metal server and it runs serveral VMs with libvirt. The real wireguard server is one of those VMs. Server B uses another socat process to unwrap the TCP-tunneled UDP packets and deliver them to the correct virtual machine running wireguard.

From what I get, UDP is unreliable of its own over the internet, and I've even added a few layers of encapsulation, but:

1. the reverse ssh tunnel is a TCP tunnel, which should be reliable instead
2. the two socat tunnels are both inside a single computer, so I can assume they are reliable
3. wireguard uses UDP and it should bundle all the reliability checks within itself

However from my client I get high packet loss percentages over my wireguard tunnel. The same ping test over other wireguard tunnels that do not need this ssh+socat workaround do work with 0% packet loss.

My question is: how do I debug this solution to find out where ping packets are being lost?

EDIT: after a few tests, I noticed that I get way higher packet loss percentages when I try to use the vpn tunnel more heavily than with the ping command alone, e.g. if I open a browser on my client pointing to one of the VMs. The ping command alone actually works nearly 100% reliable, e.g. it looses a packet in a blue moon, but with a browser loading a page it starts loosing a lot of packets (and the browser never finishes loading the page).

I see socat does have some buffers size options, but I don't know what numbers I should try, so I suspect throwing numbers in the dark won't lead me to a fast solution of the problem... anyway, while waiting for someone to chime in with an answer, I'm going to try big and small random numbers and report back: it just takes a while for each test.