As per my previous question SSHD real-ip behind haproxy - I'm now attempting to setup HAProxy to transparently pass the IPs to the backend services.
tproxy modules loaded / and supported in haproxy (I recompiled haproxy with USE_LINUX_TPROXY=1);
~$ uname -r
3.13.0-34-generic
~$ lsmod | grep -ie tprox
xt_TPROXY 17356 0
nf_defrag_ipv6 34768 2 xt_socket,xt_TPROXY
nf_defrag_ipv4 12758 2 xt_socket,xt_TPROXY
x_tables 34059 8 ip6table_filter,xt_mark,ip_tables,xt_socket,iptable_filter,xt_TPROXY,iptable_mangle,ip6_tables
~$ haproxy -vv
HA-Proxy version 1.5.3 2014/07/25
Copyright 2000-2014 Willy Tarreau <[email protected]>
Build options :
TARGET = linux2628
CPU = native
CC = gcc
CFLAGS = -O2 -march=native -g -fno-strict-aliasing
OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
server configuration;
~$ cat haproxy-iptables.sh
#!/bin/bash
sudo iptables -t mangle -N DIVERT
sudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
sudo iptables -t mangle -A DIVERT -j MARK --set-mark 111
sudo iptables -t mangle -A DIVERT -j ACCEPT
sudo ip rule add fwmark 111 lookup 100
sudo ip route add local 0.0.0.0/0 dev lo table 100
sudo ip route flush cache
~$ cat /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.ip_nonlocal_bind=1
net.ipv4.conf.lo.rp_filter=0
~$ cat /etc/haproxy/haproxy.cfg
frontend public
mode tcp
bind :80
redirect scheme https code 301 if !{ ssl_fc }
bind :443 ssl crt example.pem no-tls-tickets
tcp-request inspect-delay 5s
tcp-request content accept if HTTP
use_backend ssh if { ssl_fc_sni ssh.example.com }
backend ssh
mode tcp
source 0.0.0.0 usesrc clientip
server ssh 127.0.0.1:22
timeout server 2h
~$ tail -f /var/log/haproxy.log
Aug 18 16:07:44 localhost haproxy[3323]: 203.000.000.000:56914 [18/Aug/2014:16:07:24.233] public~ ssh/ssh 71/-1/20076 0 sC 0/0/0/0/3 0/0
without the source 0.0.0.0 usesrc clientip line in haproxy.cfg I get output in /var/log/auth.log
Aug 18 16:20:23 localhost sshd[6532]: debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8
Aug 18 16:20:23 localhost sshd[6532]: debug1: inetd sockets after dupping: 3, 3
Aug 18 16:20:23 localhost sshd[6532]: debug1: Connection refused by tcp wrapper
Aug 18 16:20:23 localhost sshd[6532]: refused connect from localhost (127.0.0.1)
when the line is added to the ssh backend block, nothing is logged in auth.log - the packets are not hitting sshd (they still are listed in haproxy.log)
I gather it's probably related to the net.ipv4 settings?
In order to be able to use TPROXY, your SSH server must use the haproxy box as router (well, the haproxy box must be in the way of the packets, but this is the easiest way to do it).
You also need the send_redirects kernel parameter enabled:
I suggest you have a look at mmproxy, that proved the easiest way for me to set this up. It's transparent for the particular daemon,