We have a Apache forward proxy sending requests to thousands of back end servers.
The servers are faulty, in that they require both the initial challenge request (without authentication) and the next request (with authentication) to be on the same http connection. We cannot get this fixed in any reasonable timescale.
So we would like the forward proxy to use the same connection, possibly via connection pooling?
Apache forward proxies , by default, closes the connection as soon as it gets a response. This is by design
“The default worker for forward proxying does not use connection pooling in the naive sense. It closes each connection after each request.
What Ryujiro Shibuya was observing was that Apache signals it would keep the connection open even in forward proxy mode, but then acually closes the connection. We are discussing a fix to this, namely always signalling "Connection: close" from the beginning for the default forward and revere proxy workers. “
There is some suggestion it can be worked around:
"You can define explicit workers though (e.g. using ProxyPass for reverse and as Rüdiger wrote likely also in forward proxy mode, which then will use HTTP Keep-Alive (by default, depending on several config options)"
But I don't know how to do this. Something with ProxySet, possibly? The issue with that is that I need to somehow specify the URLs, but this is a forward proxy - there are many possible origin servers and I cannot enumerate them up front.
How should we configure things to get this connection re-use?
(this should be a comment, but its a bit long)
And you've only just noticed its not working?
If you can handle the traffic for "thousands of servers" on a single Apache instance, it rather suggests something wrong with your architecture.
You can't rely on every component in the chain (i.e. not just your proxy) implementing keepalives in the way the back end servers seem to be expecting. So what you are attempting here is at best mitigation of an issue.
I would strongly urge you to fix the root cause here.
No. Connection pooling maintains a set of open connections to the origin server. It does not tie the client side connection to the origin server connection.
Without knowing a lot more about whats going on in the proxy server it's hard to advise what the right approach to this workaround is; while Apache makes a fabulous webserver, IMHO, there are better tools to use as proxies (ATS, haproxy, pound, squid, nginx, varnish.
From my knowledge of Apache httpd, I don't think its possible to do what you are trying to achieve.