I'm attempting to build a load-balanced SMTP cluster. The mail servers already exist and run Exim 4. Initially, I looked at using Nginx to do the load-balancing, however on the test system all the mail servers see the inbound connections as coming from the load-balancer IP rather than the actual remote sender IP, and after extensive Googling there doesn't appear to be any way round this. As that effectively turns the mail cluster into an open relay it's clearly a non-starter, which is a pity as Nginx works beautifully otherwise.
So I'm looking to use HAProxy instead, as I gather from further Googling that it has the ability to pass the connections with their original source IP intact so the system relay-allow lists and ACLs will operate correctly.
However having set HAProxy up as per several on-line examples, I either get "SMTP synchronisation error" (and a 500 series error so mail will bounce), and the connection immediately dropping, or just the connection dropping with no SMTP message at all.
Here is the haproxy.conf that's in use:
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user nobody
group nobody
daemon
stats socket /var/lib/haproxy/stats
defaults
log global
option redispatch
retries 3
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout check 10s
maxconn 3000
listen smtp
bind 0.0.0.0:25
mode tcp
no option http-server-close
balance roundrobin
# option smtpchk HELO smtp-in.example.com
server smtp01 10.0.0.141:25 send-proxy check
server smtp02 10.0.0.143:25 send-proxy check
Despite the presence of the send-proxy command, which I gather is how you tell haproxy to pass through the source IP, the Exim logs look like this:
016-12-26 07:06:48 SMTP protocol synchronization error (input sent without waiting for greeting): rejected connection from H=[10.0.0.150] input="PROXY TCP4 10.0.0.150 10.0.0.143 40334 25\r\nHELO smtp-in.example.comr\n"
In this case .150 is the load balancer and .143 is the Exim SMTP server.
Questions:
Is it in fact possible to get Nginx to present the SMTP connections to the mail servers with the source IP being the real remote source IP instead of the load-balancer?
Alternatively, is this possible in HAProxy, and how is it done?
In this case the current production system runs LVS, however that relies on both the load balancer and the loopback interfaces on all the mail servers sharing the same IP address. The new load balancer will be OpenSUSE 42.2, and amongst other things if that detects an IP already in use on the network it appears to helpfully removes it from itself to avoid a conflict. So LVS is out in the new build.
Other solutions to the problem which I'm considering include separating inbound and outbound SMTP traffic entirely (currently it all runs through the same load balancer), installing a simple relay (qmail possibly) on the load-balancer IP address, configured to only allow recognised ranges as per standard relay practice, and using simple DNS round-robin on the MX records to send inbound SMTP direct to the mail servers. But a load-balanced solution would be more elegant.
I used the custom tcp check thus