Home / server / Questions / 874859
Accepted
Micah R Ledbetter
Asked: 2017-09-22 07:23:23 +0800 CST

How can I prevent (almost) all non-VPN traffic using IPtables to a Docker service?

  • 772

I have a server that I want to be accessible on the public Internet via SSH only, but I want it to be able to connect out to a VPN, and bind services to its VPN interface.

I'm using Linode to host a Ubuntu 16.04.2 LTS VM, ZeroTier for my VPN layer, and Docker in Swarm mode (with just a single node) for the services.

Running ifconfig -a on the VM returns this (public IPs redacted for privacy):

docker0   Link encap:Ethernet  HWaddr 02:42:f7:e7:e6:1e
          inet addr:172.17.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

docker_gwbridge Link encap:Ethernet  HWaddr 02:42:ad:82:56:0f
          inet addr:172.18.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:adff:fe82:560f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6083 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16927 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6364682 (6.3 MB)  TX bytes:49725061 (49.7 MB)

eth0      Link encap:Ethernet  HWaddr f2:3c:91:a7:c3:ac
          inet addr:<REDACTED>  Bcast:<REDACTED>  Mask:255.255.255.0
          inet6 addr: <REDACTED>/64 Scope:Global
          inet6 addr: <REDACTED>/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:94675 errors:0 dropped:0 overruns:0 frame:0
          TX packets:47778 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:59390529 (59.3 MB)  TX bytes:14765979 (14.7 MB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:54171 errors:0 dropped:0 overruns:0 frame:0
          TX packets:54171 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:6243573 (6.2 MB)  TX bytes:6243573 (6.2 MB)

veth8323309 Link encap:Ethernet  HWaddr 82:b3:ec:d8:8c:9b
          inet6 addr: fe80::80b3:ecff:fed8:8c9b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:15 errors:0 dropped:0 overruns:0 frame:0
          TX packets:538 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1248 (1.2 KB)  TX bytes:22932 (22.9 KB)

veth602daf7 Link encap:Ethernet  HWaddr 82:7e:cf:b3:cb:a4
          inet6 addr: fe80::807e:cfff:feb3:cba4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3065 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11154 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6187902 (6.1 MB)  TX bytes:794193 (794.1 KB)

zt0       Link encap:Ethernet  HWaddr 42:08:c2:e8:fc:93
          inet addr:192.168.196.106  Bcast:192.168.196.255  Mask:255.255.255.0
          inet6 addr: fe80::4008:c2ff:fee8:fc93/64 Scope:Link
          inet6 addr: fc7b:7a95:194:6f84:bf9a::1/40 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:2800  Metric:1
          RX packets:442 errors:0 dropped:0 overruns:0 frame:0
          TX packets:279 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:52165 (52.1 KB)  TX bytes:46603 (46.6 KB)

My Docker compose file looks like this:

version: "3"
services:
  jenkins:
    image: jenkins/jenkins:lts
    ports:
      - "80:8080"
    volumes:
      - jenkins_home:/var/jenkins_home
    deploy:
      replicas: 1
volumes:
  jenkins_home:

However, I can't seem to get the iptables rules right. I had expected to be able to do this:

# Allow established connections, including outbound connections this server initiated, to continue to receive replies
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow SSH on eth0
iptables -A INPUT -i eth0 -p tcp --dport 22 -j ACCEPT

# Allow UDP 9993 on eth0 (required for ZeroTier)
iptables -A INPUT -i eth0 -p udp --dport 9993 -j ACCEPT
iptables -A OUTPUT -p udp --dport 9993 -j ACCEPT

# Reject everything that hasn't already been accepted (rules are applied in order)
iptables -A INPUT -i eth0 -j REJECT

However, when I deploy and start my Docker Swarm service, I end up being able to access the service via the address on eth0 from a remote host that's not on the VPN, from the VPN address on zt0 from a remote host that is on the VPN, but not from localhost on the VM itself.

What am I doing wrong here? How can I get iptables rules that will prevent traffic from the public Internet but allow it from only the ZeroTier VPN?

iptables

1 Answers

  1. Best Answer

    After more research, I found my problem.

    The gist of it is that Docker uses its own chains for forwarding packets. It will modify the DOCKER and DOCKER-INGRESS chains when the Docker daemon starts, and when it deploys new containers, but it will not modify a chain called DOCKER-USER that is intended for administrator-defined firewall rules of the type I want to apply.

    Armed with that knowledge, I was able to get what I wanted with the following iptables rules:

    #!/bin/sh

set -e
set -u

# REMINDER: DROP vs REJECT:
# a DROP operation blackholes the packet, likely causing client to wait dozens of seconds for a timeout
# a REJECT operation sends an ICMP response letting the client know the packet was rejected

# REMINDER: IPv4 vs IPv6
# `iptables` only applies IPv4 rules; `ip6tables` only lapplies IPv6 rules.

## System firewall rules

# Allow any established sessions from our container host - including replies to outbound queries - to receive traffic
iptables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED --jump ACCEPT
ip6tables --append INPUT -m conntrack --ctstate ESTABLISHED,RELATED --jump ACCEPT

# Allow SSH on eth0
iptables --append INPUT --in-interface eth0 --protocol tcp --dport 22 --jump ACCEPT
ip6tables --append INPUT --in-interface eth0 --protocol tcp --dport 22 --jump ACCEPT

# Allow UDP 9993 on eth0 (required for ZeroTier)
iptables --append INPUT --in-interface eth0 --protocol udp --dport 9993 --jump ACCEPT
ip6tables --append INPUT --in-interface eth0 --protocol udp --dport 9993 --jump ACCEPT
iptables --append OUTPUT --protocol udp --dport 9993 --jump ACCEPT
ip6tables --append OUTPUT --protocol udp --dport 9993 --jump ACCEPT

# Reject everything to eth0 that hasn't been accepted by a previous rule
iptables --append INPUT --in-interface eth0 --jump REJECT
ip6tables --append INPUT --in-interface eth0 --jump REJECT

## Docker firewall rules

# NOTE: Docker does not enable IPv6 networking by default, so there is no need for `ip6tables` here
# <https://docs.docker.com/engine/userguide/networking/default_network/ipv6/>

# NOTE: Docker promises not to modify the DOCKER-USER chain so that we can do this type of filtering

# Create the chain we need if Docker hasn't done so already
iptables --new DOCKER-USER 2>/dev/null || true

# Allow any established sessions from our containers - including replies to outbound queries - to receive traffic
iptables --append DOCKER-USER --match conntrack --ctstate ESTABLISHED,RELATED --jump ACCEPT

# Reject packets that would be forwarded to Docker when they come in over eth0 (the public interface)
iptables --append DOCKER-USER --in-interface eth0 ---jump REJECT

    Note that some of these rules are not specific to Docker, but exist to reject packets intended for my Docker server - I only want to accept VPN and SSH packets to the server itself.

    • 0