Home / server / Questions / 788562
Accepted
Necto
Asked: 2016-07-08 11:31:30 +0800 CST

Why IP masquerade (source NAT) in linux kernel fails to set source IP on high packet rates

  • 772

I am trying to test throughput of the standard linux NAT. I have setup the following virtual network:

[192.168.42.5] <--> [192.168.42.2:192.168.35.2] <--> [192.168.35.10]

The middle host has 2 interfaces, runs ubuntu 14.04 and is configured as NAT:

sysctl -w net.ipv4.ip_forward=1
echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
iptables -A FORWARD -i eth2 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i eth2 -o eht1 -j ACCEPT

The [192.168.42.5] host also has the route rule for the second network:

sudo ip route add 192.168.35.0/24 via 192.168.42.2 dev eth1

Then I run hping3 on the [192.168.42.5] host:

sudo hping3 192.168.35.10  -k -s 3453 -S -n -p 80 -i u1000

And I also watch the communication on the [192.168.35.10] with tcpdump:

sudo tcpdump -i eth1

The system works OK on the rates ~1k Pkt/second. tcpdump displays arriving packets:

IP 192.168.42.5.3453 > 192.168.35.10.http: Flags [S], seq 1520954148, win 512, length 0

But when I increase the packet rate (with -i u700 key), for some reason the NAT fails to rewrite the source address, and the target gets packets like this:

IP 192.168.42.5.3453 > 192.168.35.10.http: Flags [S], seq 1554999038, win 512, length 0

and ignores them having no available route to the 192.168.42.0/24 subnet, so hping3 does not get any response. If that happened, I need to relax and wait for about 1-2 minutes, until I can get NAT to the normal operation even on the slow packet rates.

Why does NAT fail so weirdly? If it lacks capacity, why does it stil forward unmodified packets?

P.S. When I set-up the usual router instead of the NAT (with the corresponding rule in the [192.168.35.10] host), it routinely handles even 10k Pkt/s rates.

linux nat masquerade

1 Answers

  1. Best Answer

    The issue may be the -k flag. From what I see, you set up a very unusual setup creating high volumes of connections with the same source/destination. IPtables is a statefull firewall and tracks each connection individually. I would speculate that you end up creating a high volume of collisions in a hash table. This generally slows down access which may be what you are seeing.

    The router may be operating in a stateless or semi-stateful manner. This reduces the overhead involved.

    • 3