I'm trying to setup transparent proxying networks on my host.
Real Client and Proxy targets are containters but in this experiment I use netns (network namespace) separated envinroment.
To redirect client traffic to proxy transparently, I use policy routing.
Client (C) Proxy (P)
10.10.1.1/24 10.10.2.1/24
veth0 veth0
| |
veth pair veth pair
| |
-----------(HOST)--------------
client-veth0 proxy-veth0
10.10.1.2/24 10.10.2.2/24
| | 172.16.202.30
+-----------------+-------------- enp4s0 ---- INTERNET
# Policy Routing on Host
# [Client->Proxy]
# ip rule: from 10.10.1.0/24 iif client-veth0 lookup 100
# ip route: (100) default via 10.10.2.1 dev proxy-veth0
# [Proxy->Internet]
# ip route: (master) default via 172.16.202.1 dev enp4s0 proto static metric 100
# iptables: -t nat -A POSTROUTING -s 10.10.1.1/32 -o enp4s0 -j MASQUERADE
# [Internet->Proxy]
# ip rule: from all to 10.10.1.0/24 iif enp4s0 lookup 100
# ip route: (100) default via 10.10.2.1 dev proxy-veth0
# [Proxy->Client]
# ip rule: from all to 10.10.1.0/24 iif proxy-veth0 lookup 101
# ip route: (101) default via 10.10.1.1 dev client-veth0
Problem is, When I ping 8.8.8.8 from Client, within client netns, source ip masquerading does not happen.
iptables masquerade rule does not match and defaults to ACCEPT .
I expect that tcpdump on enp4s0 shows 172.16.202.30 --> 8.8.8.8, but it shows 10.10.1.1 --> 8.8.8.8, without source IP masquerading.
I recorded pcap on internet line to clarify SNAT does not occur. client_to_goolge is recorded from separate machine outside enp4s0:
$ tcpdump -r client_to_google -n
reading from file client_to_google, link-type EN10MB (Ethernet)
23:35:40.852257 IP 10.10.1.1 > 8.8.8.8: ICMP echo request, id 14867, seq 1, length 64
23:35:41.865269 IP 10.10.1.1 > 8.8.8.8: ICMP echo request, id 14867, seq 2, length 64
When I checked on iptables mangle table, packets flows by given policy:
PREROUTING: client-veth0, 10.10.1.1 --> 8.8.8.8
POSTROUTING: proxy-veth0, 10.10.1.1 --> 8.8.8.8
PREROUTING: proxy-veth0, 10.10.1.1 --> 8.8.8.8
POSTROUTING: enp4s0, 10.10.1.1 --> 8.8.8.8
However, when I change masquerade rule on proxy-veth0 out interface, like this iptables: -t nat -A POSTROUTING -s 10.10.10.1/32 -o proxy-veth0 -j MASQUERADE, masquerading happens. That is
10.10.2.2 --> 8.8.8.8 packets are captured.
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
...
11 0 0 MASQUERADE all -- * enp4s0 10.10.1.1 0.0.0.0/0
12 1 84 MASQUERADE all -- * proxy-veth0 10.10.1.1 0.0.0.0/0
Above table shows that rule #11 enp4s0 output condition did not trigger. Rule #12 was inserted after several test with rule #11. Rule #12 shows that proxy-veth0 output condition did trigger. Are there any differences between enp4s0 master nic and proxy-veth0 virthual interface with iptables?
Any comments will be appreciated deeply, Thank you.
I have to assume that the transparent proxy is acting as a router, at least for ICMP, so will route back the ICMP echo where it came from (veth0).
Finding the problem
When reproducing your setup and witnessing your problem, I added a TRACE on the host using iptables (legacy, which might have slight differences with iptables-nft's version) like this (I also forced the creation of the filter table (
iptables -S) to have it in the traces):And a single ping shows in kernel logs (hint, if host isn't the actual initial host:
sysctl -w net.netfilter.nf_log_all_netns=1):At the same time, having
conntrack -Erunning on the host shows the matching:What happened:
Since this conntrack's limitation hindered some use cases in the past, like yours, an additional feature was added:
conntrack zones
It allows to sort-of duplicate the conntrack facility, including NAT handling, but has to be done manually and match the problem: here the routing topology.
So here the client <-> proxy traffic, in conntrack's point of view, must be split from other traffic.
I would have preferred to also split the proxy <-> Internet traffic from the generic host traffic, but this is too difficult, because the raw table, where zones must be assigned to a packet, sees only the non-de-NATed traffic, so Internet replies will all arrive with destination 172.16.202.30). Anyway There's no duplicated flow here between both like with the client <-> proxy flow, so that's not really needed.
zone 0 (0 means no special zone): generic host traffic along with proxy <-> Internet traffic.
Nothing special to do, this is the default.
zone 1: client <-> proxy traffic. The
CT --zonetarget is used. The value here is chosen arbitrarily and not needed anywhere else for this case.The correct results (I merged both tools' outputs) are now:
Here a single first packet from a new flow triggers twice iptables' nat table, the first time with no effect. Actually conntrack considers there are two flows, because the first flow has the additional attribute
zone=1.This is normal iptables NAT behavior. Testing right now with tcpdump, I use three virtual machines, on every virtual machine i set SNAT rule
vm1(10.10.1.1) ---> vm2(10.10.2.2) ---> vm3(1.2.3.4) (public ip address)
Then I try ping 8.8.8.8 from 10.10.1.1, run tcpdump on all three machines and results was next: on vm2 10.10.1.1 ---> 8.8.8.8 on vm3 10.10.2.2 ---> 8.8.8.8
In theory if I have vm4 i would see 1.2.3.4 ---> 8.8.8.8
Could You please show your proxy config?
Boris