Home / server / Questions / 1098041
Accepted
Tuinslak
Asked: 2022-04-08 04:47:41 +0800 CST

Using iptables forwarding, while properly keeping source IP

  • 772

I have a server running Wireguard (thus needing masquerade) and a container running on port 2525.

I have the following iptables rules:

iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 25 -j DNAT --to-destination 172.18.0.1:2525
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

When connecting to server:2525 directly, the Docker container is able to see my real IP address (1.2.3.4). When connecting to port server:25, the Docker container is seeing the local IP that's provided by docker network:

Apr 07 12:45:46 mx postfix/smtpd[87]: lost connection after CONNECT from unknown[172.18.0.1]
Apr 07 12:45:46 mx postfix/smtpd[87]: disconnect from unknown[172.18.0.1] commands=0/0

How do I make sure the Docker container is properly seeing the public IP address while connecting to port 25 (and not only when connecting to port 2525).

Thanks

# iptables -L -n -v -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
52300 3131K DNAT       tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:25 to:172.18.0.1:2525
 150K 8524K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    2   120 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 3385  256K MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
1733K  104M MASQUERADE  all  --  *      !br-b147ffdbc9f3  172.18.0.0/16        0.0.0.0/0
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.2           172.17.0.2           tcp dpt:53
    0     0 MASQUERADE  udp  --  *      *       172.17.0.2           172.17.0.2           udp dpt:53
    0     0 MASQUERADE  tcp  --  *      *       172.18.0.2           172.18.0.2           tcp dpt:25

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
   12  1419 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  br-b147ffdbc9f3 *       0.0.0.0/0            0.0.0.0/0
   56  3192 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:5354 to:172.17.0.2:53
    0     0 DNAT       udp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:5354 to:172.17.0.2:53
  107  6020 DNAT       tcp  --  !br-b147ffdbc9f3 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:2525 to:172.18.0.2:25

# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 32:d0:56:15:0a:64 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 159.223.80.86/20 brd 159.223.95.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 10.15.0.19/16 brd 10.15.255.255 scope global eth0:1
       valid_lft forever preferred_lft forever
    inet6 2400:6180:0:d0::f57:6001/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::30d0:56ff:fe15:a64/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 32:dc:4a:e4:27:be brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    altname ens4
    inet 10.130.244.15/16 brd 10.130.255.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::30dc:4aff:fee4:27be/64 scope link
       valid_lft forever preferred_lft forever
4: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 10.200.200.52/24 scope global wg0
       valid_lft forever preferred_lft forever
5: wg1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 10.222.111.1/24 scope global wg1
       valid_lft forever preferred_lft forever
6: br-b147ffdbc9f3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:46:21:70:c0 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 brd 172.18.255.255 scope global br-b147ffdbc9f3
       valid_lft forever preferred_lft forever
    inet6 fe80::42:46ff:fe21:70c0/64 scope link
       valid_lft forever preferred_lft forever
7: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:66:22:41:91 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:66ff:fe22:4191/64 scope link
       valid_lft forever preferred_lft forever
9: veth31eff9d@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP group default
    link/ether e6:fb:80:5d:c7:a3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::e4fb:80ff:fe5d:c7a3/64 scope link
       valid_lft forever preferred_lft forever
19: veth01269f5@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-b147ffdbc9f3 state UP group default
    link/ether 36:f4:e7:43:5f:da brd ff:ff:ff:ff:ff:ff link-netnsid 2
    inet6 fe80::34f4:e7ff:fe43:5fda/64 scope link
       valid_lft forever preferred_lft forever
port-forwarding iptables forwarding docker

1 Answers

  1. Best Answer

    Just let Docker handle the redirection, which is dynamic and could change when containers are added, removed or restarted. But see UPDATE below.

    This redirection should not be to 172.18.0.1 which is the host and not the container. When the host receives such connection, it is handled by docker-proxy which proxies it to the container, losing the source IP address in the process.

    Docker already DNAT + routes this port correctly (except from the host itself, where docker-proxy plays this role) in the very last rule of the ruleset, to the running container with the 172.18.0.2 address. Except it's configured to use port 2525 rather than port 25.

      107  6020 DNAT       tcp  --  !br-b147ffdbc9f3 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:2525 to:172.18.0.2:25

    This should be fixed with Docker settings, not manual iptables rules that can't change whenever container layout changes. As port 25 is privileged, if Docker is running rootless, additional settings are needed, check the documentation about exposing privileged ports.

    UPDATE (factoring in OP's comments): OP can't currently use -p 25:25 because docker-proxy clashes with the local host's SMTP server and competes for listening on port 25 on the host. That's the reason the initial (wrong) iptables redirection was made by OP.

    One can either:

    • disable globally docker-proxy by running dockerd with the property userland-proxy set to false

      either as a parameter --userland-proxy=false or as a property "userland-proxy": false added to /etc/docker/daemon.json.

      This will then allow to use docker run ... -p 25:25 ... (as documented) without clash: the host will reach itself when reaching localhost or $HOSTNAME, remote systems will reach the container and no "Address already in use" will make the host's SMTP daemon or Docker's container fail at start.

    • or else add a manual redirection (with lengthy setup below to do it almost automatically)

      Whenever the container is restarted, there is a risk that its (internal) IP address will change. So this must be computed. So with a container named mx using a network named mx and a single IP address involved this can be done as explained below.

      Create a separate prerouting chain (so it can be flushed without having to flush anything else) and call it first:

      iptables -t nat -N mynat
iptables -t nat -I PREROUTING -j mynat

      The container's IP address can be retrieved programmatically (for the simple case of a container named mx with a single address):

      containerip=$(docker container inspect --format '{{.NetworkSettings.IPAddress}}' mx)

      (or one could use jq: containerip=$(docker container inspect mx | jq '.[].NetworkSettings.IPAddress')

      Finding the bridge interface name is more convoluted, or at least I couldn't find a way for it using only docker ... inspect. So find its IP address on the host and query the host using ip address to find only the bridge interface where this specific IP address is set (requires the jq command.)

       bridgeip=$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' mx)
 bridgeinterface=$(ip -json address show to "$bridgeip"/32 | jq -r '.[].ifname')

      Flush and repopulate mynat each time the container is (re)started:

      iptables -t nat -F mynat
iptables -t nat -A mynat ! -i "$bridgeinterface" -p tcp --dport 25 -j DNAT --to-destination "$containerip":25

      which would be for the current case:

      iptables -t nat -I mynat ! -i br-b147ffdbc9f3 -p tcp --dport 25 -j DNAT --to-destination 172.18.0.2:25

      And to be sure Docker's own firewalling rules don't block such traffic, do something similar starting in filter/FORWARD from the DOCKER-USER chain.

      Initially (if from boot, you might have to also create first DOCKER-USER):

      iptables -N myforward
iptables -I DOCKER-USER 1 -j myforward

      Then later each time the container is (re)started:

      iptables -F myforward
iptables -A myforward -p tcp ! -i "$bridgeinterface" -d "$containerip" -p tcp --dport 25 -j ACCEPT

      which would be for the current case:

      iptables -A myforward -p tcp ! -i br-b147ffdbc9f3 -d 172.18.0.2 -p tcp --dport 25 -j ACCEPT

    Notes:

    • To simplify rules above and avoid some of the calculations, the container and its bridge network can be started with fixed IP addresses. See for example this SO Q/A: Assign static IP to Docker container.

    • Here's also an UL SE Q/A with an answer of mine about problems when interacting with Docker (it's geared for nftables but some parts about the DOCKER-USER chain or br_netfilter bridge interactions are still of interest): nftables whitelisting docker

    • 2