I have an NGINX server being used as a TCP load balancer. It is default to round-robin load balancing, so my expectation is that for a given client IP, every time they hit the endpoint they will get a different backend upstream server for each request. But instead what is happening is that they get the same upstream server every time, and each distinct client IP is getting a distinct upstream server. This is bad because my clients generate a lot of traffic and it is causing hotspots because any given client can only utilize one upstream server. It seems to slowly rotate a given client IP across the upstream servers; again I want it to randomly assign each request to an upstream per request.
How can I make NGINX randomely assign the upstream server for every request? I tried the random keyword and this had no effect. Any help would be greatly appreciated.
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
stream {
upstream api_backend_http {
server node1.mydomain.com:80;
server node2.mydomain.com:80;
server node6.mydomain.com:80;
server node14.mydomain.com:80;
server node18.mydomain.com:80;
server node19.mydomain.com:80;
server node21.mydomain.com:80;
server node22.mydomain.com:80;
server node24.mydomain.com:80;
}
upstream api_backend_https {
server node1.mydomain.com:443;
server node2.mydomain.com:443;
server node6.mydomain.com:443;
server node14.mydomain.com:443;
server node18.mydomain.com:443;
server node19.mydomain.com:443;
server node21.mydomain.com:443;
server node22.mydomain.com:443;
server node24.mydomain.com:443;
}
server {
listen 80;
proxy_pass api_backend_http;
proxy_buffer_size 16k;
proxy_connect_timeout 1s;
}
server {
listen 443;
proxy_pass api_backend_https;
proxy_buffer_size 16k;
proxy_connect_timeout 1s;
}
}
Because you should stop using nginx as a TCP load balancer for other Web-servers and switch it to a full-fledged HTTP reverse-proxy, which it is. That way you will get the per-request RR, which you want (with persistent connections disabled by default), instead of TCP session distribution.