Home / server / Questions / 34552
In Process
ahanson
Asked: 2009-07-02 08:01:45 +0800 CST

Is there a name based virtual host SSH reverse proxy?

  • 772

I've grown quite fond of HTTP reverse proxies in our development environment and found the DNS based virtual host reverse proxy quite useful. Having only one port (and the standard one) open on the firewall makes it much easier for management.

I'd like to find something similar to do SSH connections but haven't had much luck. I'd prefer not to simply use SSH tunneling since that requires opening port ranges other than the standard. Is there anything out there that can do this?

Could HAProxy do this?

ssh reverse-proxy

7 Answers

  1. I don't believe name-based SSH is something that will be possible given how the protocol works.

    Here are some alternatives.

    • You could do is setup the host that answers for port 22 to act as a gateway. Then you can configure the ssh server to forward requests to the inside based on the key. SSH Gateway example with keys

    • You could adjust your client to use that host as a proxy. That is, it would ssh to the gateway host, and then make use that host to make a connection to the internal host. SSH proxy with client configuration.

    • You could also setup a simple http proxy on the edge. Then use that to allow incoming connections. SSH via HTTP proxy.

    Obviously with all the above, making sure you properly configure and lock down the gateway is pretty important.

    • 26

  2. I have been searching for a solution for this problem on and off for the last 16 months. But each time I look, it seems impossible to do this with the SSH protocol as specified in relevant RFCs and implemented by major implementations.

    However if you are willing to use a slightly modified SSH client and you are willing to utilize protocols in way that was not exactly intended when they were designed, then it is possible to achieve. More on this below.

    Why it is not possible

    The client does not send the hostname as part of the SSH protocol.

    It might send the hostname as part of a DNS lookup, but that might be cached, and the path from client through resolvers to authoritative servers might not cross the proxy, and even if it did there is no robust way of associating specific DNS lookups with specific SSH clients.

    There is nothing fancy you can do with the SSH protocol itself either. You have to pick a server without even having seen the SSH version banner from the client. You have to send a banner to the client, before it will send anything to the proxy. The banners from the servers could be different, and you have no chance of guessing, which one is the correct one to use.

    Even though this banner is sent unencrypted, you cannot modify it. Every bit of that banner will be verified during connection setup, so you'd be causing a connection failure a bit down the line.

    The conclusion to me is pretty clear, one has to change something on the client side in order to make this connectivity work.

    Most of the workarounds are encapsulating the SSH traffic inside a different protocol. One could also imagine an addition to the SSH protocol itself, in which the version banner send by the client include the hostname. This can remain compatible with existing severs, since part of the banner is currently specified as a free form identification field, and though clients typically wait for the version banner from the server before sending their own, the protocol does permit the client to send their banner first. Some recent versions of the ssh client (for example the one on Ubuntu 14.04) does send the banner without waiting for the server banner.

    I don't know of any client, which has taken steps to include the hostname of the server in this banner. I have send a patch to the OpenSSH mailing list to add such a feature. But it was rejected based on a desire to not reveal the hostname to anybody snooping on the SSH traffic. Since a secret hostname is fundamentally incompatible with the operation of a name based proxy, don't expect to see an official SNI extension for the SSH protocol anytime soon.

    A real solution

    The solution that worked best for me was actually to use IPv6.

    With IPv6 I can have a separate IP address assigned to each server, so the gateway can use the destination IP address to find out which server to send the packet to. The SSH clients might sometimes be running on networks where the only way to get an IPv6 address would be by using Teredo. Teredo is known to be unreliable, but only when the native IPv6 end of the connection is using a public Teredo relay. One can simply put a Teredo relay on the gateway, where you'd run the proxy. Miredo can be installed and configured as a relay in less than five minutes.

    A workaround

    You can use a jump host/bastion host. This approach is intended for cases where you don't want to expose the SSH port of the individual servers directly to the public internet. It does have the added benefit on reducing the number of externally facing IP address you need for SSH, which is why it is usable in this scenario. The fact that it is a solution intended to add another layer of protection for security reasons doesn't prevent you from using it when you don't need the added security.

    ssh -o ProxyCommand='ssh -W %h:%p user1@bastion' user2@target

    A dirty to hack to make it work if the real solution (IPv6) is outside of your reach

    The hack I am about to describe should only be used as the absolutely last resort. Before you even think about using this hack, I strongly recommend getting an IPv6 address for each of the servers which you want to be externally reachable through SSH. Use IPv6 as your primary method for accessing your SSH servers and only use this hack when you need to run an SSH client from an IPv4-only network where you don't have any influence on getting IPv6 deployed.

    The idea is that traffic between client and server need to be perfectly valid SSH traffic. But the proxy only need to understand enough about the stream of packets to identify the hostname. Since SSH doesn't define a way to send a hostname, you can instead consider other protocols which do provide such a possibility.

    HTTP and HTTPS both allow for the client to send a hostname before the server has send any data. The question now is, whether it is possible to construct a stream of bytes which is simultaneously valid as SSH traffic and as either HTTP and HTTPS. HTTPs it is pretty much a non-starter, but HTTP is possible (for sufficiently liberal definitions of HTTP).

    SSH-2.0-OpenSSH_6.6.1 / HTTP/1.1
$: 
Host: example.com

    Does this look like SSH or HTTP to you? It is SSH and completely RFC compliant (except some of the binary characters got mangled a bit by the SF rendering).

    The SSH version string includes a comment field, which in the above has the value / HTTP/1.1. After the newline SSH has some binary packet data. The first packet is a MSG_SSH_IGNORE message send by the client and ignored by the server. The payload to be ignored is:

    : 
Host: example.com

    If an HTTP proxy is sufficiently liberal in what it accepts, then the same sequence of bytes would be interpreted as an HTTP method called SSH-2.0-OpenSSH_6.6.1 and the binary data at the start of the ignore message would be interpreted as an HTTP header name.

    The proxy would understand neither the HTTP method or the first header. But it could understand the Host header, which is all it needs in order to find the backend.

    In order for this to work the proxy would have to be designed on the principle that it only needs to understand enough HTTP to find the backend, and once the backend has been found the proxy will simply pass the raw byte stream and leave the real termination of the HTTP connection to be done by the backend.

    It may sound like a bit of a stretch to make so many assumptions about the HTTP proxy. But if you were willing to install a new piece of software developed with the intention to support SSH, then the requirements for the HTTP proxy don't sound too bad.

    In my own case I found this method to work on an already installed proxy with no changes to code, configuration, or otherwise. And this was for a proxy written with only HTTP and no SSH in mind.

    Proof of concept client and proxy. (Disclaimer that proxy is a service operated by me. Feel free to replace the link once any other proxy has been confirmed to support this usage.)

    Caveats of this hack

    • Don't use it. It's better to use the real solution, which is IPv6.
    • If the proxy attempts to understand the HTTP traffic, it is surely going to break.
    • Relying on a modified SSH client isn't nice.
    • 25

  3. There is a new kid on the block. SSH piper will route your connection based on pre-defined usernames, that should be mapped to the inner hosts. This is the best solution in the context of of reverse-proxy.

    SSh piper on github

    My first tests confirmed, SSH and SFTP both work.

    • 4

  4. I don't believe this is something that would be possible, at least the way you described, although I would love to be proved wrong. It doesn't appear that the client sends the hostname that it wishes to connect to (at least in the clear). The first step of the SSH connection seems to be to set up encryption.

    In addition, you would have issues with host key verification. SSH clients will verify keys based on an IP address as well as a hostname. You would have multiple hostnames with different keys, but the same IP you're connecting to.

    A possible solution would be to have a 'bastion' host, where clients can ssh in to that machine, get a normal (or restricted if desired) shell, and can then ssh into internal hosts from there.

    • 3

  5. I am wondering if Honeytrap (the low interaction honeypot) that has a proxy mode couldn't be modified to achieve that.

    This honeypot is able to forward any protocol to another computer. Adding a name based vhost system (as implemented by Apache) could make it a perfect reverse proxy for any protocol no ?

    I do not have the skills to achieve that but maybe it could be a great project.

    • 2

  6. because of how ssh works I think it's not possible. Similar to https you would have to have different (external) IPs for different hosts, because the gateway doesn't know where do you want to connect to because everything in ssh is encryted

    • 1

  7. You can use HAProxy on the server side and ssh + openssl on the client side to SSH over HTTPS. This approach uses HTTPS SNI to identify the destination server.

    Example client command:

    ssh -o ProxyCommand="openssl s_client -quiet -connect <proxy IP>:2222 -servername server1" dummyName1

    dummyName1 in the above command is used purely as a placeholder since ssh requires a hostname, but openssl is doing all of the connection work here

    Basic HAProxy config:

    frontend fe_ssh
   bind *:2222 ssl crt /etc/haproxy/certs/ssl.pem
   mode tcp
   log-format "%ci:%cp [%t] %ft %b/%s %Tw/%Tc/%Tt %B %ts %ac/%fc/%bc/%sc/%rc %sq/%bq dst:%[var(sess.dst)] "
   tcp-request content set-var(sess.dst) ssl_fc_sni
   use_backend %[ssl_fc_sni]

backend server1
   mode tcp
   server s1 <backend ip 1>:22 check

backend server2
   mode tcp
   server s2 <backend ip 2>:22 check

backend server3
   mode tcp
   server s2 <backend ip 3>:22 check

    More details and security improvements can be found at the Source

    • 0