Home / server / Questions / 301221
Accepted
Chris Lercher
Asked: 2011-08-16 06:35:52 +0800 CST

haproxy + stunnel + keep-alive?

  • 772

I'd like to put stunnel in front of haproxy 1.4 to handle HTTPS traffic. I also need stunnel to add the X-Forwarded-For header. This can be achieved by the "stunnel-4.xx-xforwarded-for.diff" patches from the haproxy website.

However, the description mentions:

Note that this patch does not work with keep-alive, ...

My question is: What will this mean in practice for me? I'm unsure,

  1. if this is about the keep-alive between
    • client and stunnel
    • stunnel and haproxy
    • or haproxy and backend server?
  2. what this means for performance: If I have 100 icons on a web page, will the browser have to negotiate 100 full SSL connections, or can it re-use the SSL connection, just creating new TCP connections?
ssl tcp https haproxy stunnel

4 Answers

  1. Best Answer

    This is about HTTP keep-alive, which allows for multiple resource requests to come through a single TCP session (and, with SSL, a single SSL session). This is of great importance to the performance of an SSL site, as without keep-alive, an SSL handshake would be needed for each requested resource.

    So, the concern here is one big keep-alive session from the client all the way to the backend server. It's an important thing for performance, and taken as a matter of course for modern HTTP servers, but this patch says it doesn't support it. Let's look into why..

    A keep-alive session is just more requests one after another - once the server finishes its response to one request, the server doesn't sent a FIN packet to end the TCP session; the client can simply send another batch of headers.

    To understand what that patch is doing, here's an example of a keep-alive conversation:

    Client:

    GET / HTTP/1.1
Connection: keep-alive
Host: domain.com
...

    Server:

    HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Server: Apache
Content-Length: 34
.... (other headers)
<html><head>content!</head></html>

    Here's where a non-keep-alive connection would stop. But, keep-alive allows the client to just fire off another:

    GET /images/some/image.on.the.page.jpg HTTP/1.1
Connection: keep-alive
Host: domain.com
...

    For client ID in proxying, some reverse proxies can add in the X-Forwarded-For header in each client request. That tells the upstream server where the request is coming from (instead of every request initiating from the reverse proxy's IP), for sanity in logging and other application needs.

    The X-Forwarded-For header needs to be injected into each and every client resource request sent through the keep-alive connection, as the full headers are sent each time; handling of the X-Forwarded-For header and translation into it being the "real" request IP is done on a per-request, not per-TCP-keep-alive-session, basis. And hey, maybe there's some awesome reverse proxy software out there that uses a single keep-alive session to service requests from multiple clients.

    This is where this patch fails.

    The patch at that site watches the TCP session's buffer for the end of the first set of HTTP headers in the stream, and injects the new header into the stream after the end of that first set of headers. After this is done, it considers the X-Forwarded-For job done, and stops scanning for the end of new sets of headers. This method has no awareness of all of future headers coming in via subsequent requests.

    Can't really blame them; stunnel wasn't really built to do handling and translation of the contents of its streams.

    The effect that this would have on your system is that the first request of a keep-alive stream will get the X-Forwarded-For header injected properly, and all of the subsequent requests will work just fine - but they won't have the header.

    Unless there's another header injection patch out there that can handle multiple client requests per connection (or get this one tweaked with the help of our friends over at Stack Overflow), you may need to look at other options for your SSL termination.

    • 12

  2. STunnel 4.45 fixes this properly using some new capabilities (proxy protocol) coming with HAProxy 1.15

    It also fixes the issues with previous patches and Keep Alive

    • 5

  3. Similar to what I posted in another thread, HAProxy does support native SSL on both sides since 1.5-dev12. So having X-Forwarded-For, HTTP keep-alive as well as a header telling the server that the connection was made over SSL is as simple as the following :

    listen front
    bind :80
    bind :443 ssl crt /etc/haproxy/haproxy.pem
    mode http
    option http-server-close
    option forwardfor
    reqadd X-Forwarded-Proto:\ https if { is_ssl }
    server srv1 1.1.1.1:80 check ...
    ...

    It's much easier than patching stunnel and much better than having to drop keep-alive.

    • 3

  4. Extending the excelent answer from Shane, you could use Nginx as SSL terminator in front of HAproxy. It correctly handles keep-alive between client and nginx which is the most latency sensitive side and makes a new connection to backend for each client request, sending the X-FORWARDED-FOR in each one.

    • 2