As per my previous question SSHD real-ip behind haproxy - I'm now attempting to setup HAProxy to transparently pass the IPs to the backend services.
tproxy modules loaded / and supported in haproxy (I recompiled haproxy with USE_LINUX_TPROXY=1);
~$ uname -r
3.13.0-34-generic
~$ lsmod | grep -ie tprox
xt_TPROXY 17356 0
nf_defrag_ipv6 34768 2 xt_socket,xt_TPROXY
nf_defrag_ipv4 12758 2 xt_socket,xt_TPROXY
x_tables 34059 8 ip6table_filter,xt_mark,ip_tables,xt_socket,iptable_filter,xt_TPROXY,iptable_mangle,ip6_tables
~$ haproxy -vv
HA-Proxy version 1.5.3 2014/07/25
Copyright 2000-2014 Willy Tarreau <[email protected]>
Build options :
TARGET = linux2628
CPU = native
CC = gcc
CFLAGS = -O2 -march=native -g -fno-strict-aliasing
OPTIONS = USE_LINUX_TPROXY=1 USE_ZLIB=1 USE_OPENSSL=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity, deflate, gzip
Built with OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
Running on OpenSSL version : OpenSSL 1.0.1f 6 Jan 2014
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.31 2012-07-06
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
server configuration;
~$ cat haproxy-iptables.sh
#!/bin/bash
sudo iptables -t mangle -N DIVERT
sudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
sudo iptables -t mangle -A DIVERT -j MARK --set-mark 111
sudo iptables -t mangle -A DIVERT -j ACCEPT
sudo ip rule add fwmark 111 lookup 100
sudo ip route add local 0.0.0.0/0 dev lo table 100
sudo ip route flush cache
~$ cat /etc/sysctl.conf
net.ipv4.ip_forward=1
net.ipv4.ip_nonlocal_bind=1
net.ipv4.conf.lo.rp_filter=0
~$ cat /etc/haproxy/haproxy.cfg
frontend public
mode tcp
bind :80
redirect scheme https code 301 if !{ ssl_fc }
bind :443 ssl crt example.pem no-tls-tickets
tcp-request inspect-delay 5s
tcp-request content accept if HTTP
use_backend ssh if { ssl_fc_sni ssh.example.com }
backend ssh
mode tcp
source 0.0.0.0 usesrc clientip
server ssh 127.0.0.1:22
timeout server 2h
~$ tail -f /var/log/haproxy.log
Aug 18 16:07:44 localhost haproxy[3323]: 203.000.000.000:56914 [18/Aug/2014:16:07:24.233] public~ ssh/ssh 71/-1/20076 0 sC 0/0/0/0/3 0/0
without the source 0.0.0.0 usesrc clientip line in haproxy.cfg I get output in /var/log/auth.log
Aug 18 16:20:23 localhost sshd[6532]: debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8
Aug 18 16:20:23 localhost sshd[6532]: debug1: inetd sockets after dupping: 3, 3
Aug 18 16:20:23 localhost sshd[6532]: debug1: Connection refused by tcp wrapper
Aug 18 16:20:23 localhost sshd[6532]: refused connect from localhost (127.0.0.1)
when the line is added to the ssh backend block, nothing is logged in auth.log - the packets are not hitting sshd (they still are listed in haproxy.log)
I gather it's probably related to the net.ipv4 settings?
