I have a router configured to send all the incoming traffic to a single reverse proxy machine.
This machine hosts Nginx which forwards the traffic to different servers, based on the domain name. For instance, http://a.example.com/ is forwarded to server A, and http://b.example.com/ is redirected to server B.
How do I do the same thing for Remote Desktop connections? In other words, if I have servers C and D hosting Windows, how do I connect, outside the network, to server C through c.example.com:3389 and to D through d.example.com:3389?
Just to be different, Microsoft calls this an "RDP gateway", not a proxy server. I found this guide to setting one up here.
As John Cave said, you could set up a remote desktop gateway. With this, you specify your connection with the internal name of the machine, and then use the RD Gateway server under the gateway settings of the connection. Alternatively, if you're just looking at a couple of machines, you could use port 3389 for C, and then a different port number for D.