We run haproxy as load balancer and to provide backup functionality for our API service.
Our API is a Java Application using the Jersey framework running in Apache Tomcat. Some time ago we ran into a bad performance problem on the side of our API and we just couldn't figure out what was going on. After digging deep we found that the option httpchk was causing the problem:
option httpchk GET /resource/healthcheck
http-check expect rstatus (2|3)[0-9][0-9]
server server1_8585 x.x.x.x:8585 check inter 1s fastinter 2s fall 2 rise 1 pool-purge-delay 30s
The packets send by haproxy did not include a Host header and because of that Tomcat tries to resolve the host which leads to terrible performance issues. Because of that we tuned the backend configuration to this:
The backend configuration looks like:
backend api_example_com
balance roundrobin
option httpchk GET /resource/healthcheck HTTP/1.1\r\nHost:\ server1.example.com
http-check expect rstatus (2|3)[0-9][0-9]
server server1_8585 x.x.x.x:8585 check inter 10s fastinter 5s fall 2 rise 1 pool-purge-delay 30s
server server2_8585 y.y.y.y:8585 check inter 10s fastinter 5s fall 2 rise 1 pool-purge-delay 30s backup
server server3_8585 z.z.z.z:8585 check inter 10s fastinter 5s fall 2 rise 1 pool-purge-delay 30s backup
But now we have the problem that the backup APIs get a wrong Host name as well if health-checked.
Any idea how to set up a separate health check per server?
Well you can try to create a proxy per backup server and use the track feature.
But as Gerard written it would be better to have on every server the same host name. For tomcat is this Host Name Aliases
HAProxy track suggestion (untested)