Home / server / Questions / 979236
Accepted
Tombart
Asked: 2019-08-15 04:13:24 +0800 CST

Docker packets are not being masqueraded (desipite NAT rules)

  • 772

On a machine with Debian 9 (Linux kernel 4.9) I have a Docker (18.06.1) with some containers in brigde mode. For some strange reason some packets from Docker manage to bypass MASQUERADE rule, enp2s0 is a public interface (Docker uses docker0 interface with 172.17.0.1).

$ tcpdump -vvlnn -i enp2s0 port 3000 and src net 172.16.0.0/12
tcpdump: listening on enp2s0, link-type EN10MB (Ethernet), capture size 262144 bytes
11:57:49.918655 IP (tos 0x0, ttl 63, id 62271, offset 0, flags [DF], proto TCP (6), length 52)
    172.17.0.2.55664 > x.x.x.x.3000: Flags [F.], cksum 0xe40c (correct), seq 9863202, ack 476959401, win 856, options [nop,nop,TS val 1382910659 ecr 2481487487], length 0
11:57:50.126683 IP (tos 0x0, ttl 63, id 62272, offset 0, flags [DF], proto TCP (6), length 52)
    172.17.0.2.55664 > x.x.x.x.3000: Flags [F.], cksum 0xe3d8 (correct), seq 0, ack 1, win 856, options [nop,nop,TS val 1382910711 ecr 2481487487], length 0
11:57:50.546660 IP (tos 0x0, ttl 63, id 62273, offset 0, flags [DF], proto TCP (6), length 52)
    172.17.0.2.55664 > x.x.x.x.3000: Flags [F.], cksum 0xe36f (correct), seq 0, ack 1, win 856, options [nop,nop,TS val 1382910816 ecr 2481487487], length 0

NAT rules from iptables-save:

*nat
:PREROUTING ACCEPT [11397418:724275374]
:INPUT ACCEPT [39095:3038067]
:OUTPUT ACCEPT [1328340:79997617]
:POSTROUTING ACCEPT [5102467:306147980]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -o enp2s0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 5501 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 5500 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 3000 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 48842 -j DNAT --to-destination 172.17.0.3:5501
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 48841 -j DNAT --to-destination 172.17.0.3:5500
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 13119 -j DNAT --to-destination 172.17.0.2:3000

I've tried to add MANGLE rules to catch those packets, but so far without any success:

*mangle
:PREROUTING ACCEPT [44457014385:7315518035795]
:INPUT ACCEPT [404840097:241773793538]
:FORWARD ACCEPT [44052174279:7073744241603]
:OUTPUT ACCEPT [526370610:171137381220]
:POSTROUTING ACCEPT [44578544703:7244881613871]
:bogus - [0:0]
:spoofing - [0:0]
-A PREROUTING -s 192.168.0.0/24 -i enp2s0 -j spoofing
-A PREROUTING -s 10.0.0.0/8 -i enp2s0 -j spoofing
-A PREROUTING -s 172.16.0.0/12 -i enp2s0 -j spoofing
-A PREROUTING -s 127.0.0.0/8 ! -i lo -j spoofing
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j bogus
-A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j bogus
-A PREROUTING -p tcp -m tcp --tcp-flags FIN,RST FIN,RST -j bogus
-A bogus -j LOG --log-prefix "BOGUS: "
-A bogus -j DROP
-A spoofing -j LOG --log-prefix "IP SPOOF: "
-A spoofing -j DROP
COMMIT

Any idea how can I block those packets?

Forwarded packets:

iptables -vnL FORWARD
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  44G 7074G DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  44G 7074G DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
  16G 4358G ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
  54M 3269M DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
  28G 2712G ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            state INVALID
    0     0 ACCEPT     all  --  docker0 enp2s0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  enp2s0 docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0            LOG flags 0 level 4 prefix "fw forward drop "
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state NEW

Forward rules (partially injected by Docker):

-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -m state --state INVALID -j DROP
-A FORWARD -i docker0 -o enp2s0 -j ACCEPT
-A FORWARD -i enp2s0 -o docker0 -j ACCEPT

Also OUTPUT chain should be dropping invalid packets:

-A OUTPUT -m state --state INVALID -j DROP
iptables

1 Answers

  1. Best Answer

    Likely these packets have the INVALID conntrack state. Try to add a rule into the filter/FORWARD chain to drop it.

    iptables -I FORWARD -m conntrack --ctstate INVALID -j DROP

    Also, the nat targets handle connection only from first to last packets. It means, if you add or remove the nat rule, this rule will affects only for new connections, but the existed connections (established) won't be affected.

    • 0