Ubuntu Router with different LAN subnets

After much toil trouble and a six pack I have figured out the answer to my query.

First to re-iterate: I have a home built 18.04 UBUNTU router with 3 NICS - I am posting this because 18.04 uses netplan and I did not find a suitable answer on the message boards for netplan.

NIC 1: built in gigagbit ethernet - WAN

NIC 2: 2 port 10G NIC - Bonded and connected to a layer 3 switch (located at 192.168.0.254) - subnet 192.168.0.x

NIC 3: 4 port gigabit NIC used for local internet access (separate from the switch) - subnet 192.168.1.x

As you can see, I wished to create 2 subnets - 192.168.0.x for the switch and 192.168.1.x for the 4 port NIC (to use even when the switch was disconnected).

As this machine is a router, it needs to both route, assign IP addresses (DHCP), and provide DNS. Below, you will find my configuration for netplan and DHCP. DHCP occurs on both subnets, and NIC 3 has internet access even if the switch is disconnected from NIC 2 (ie it bypasses the switch and uses the router as a gateway)

As you look at the configuration, you will notice 2 bridges:

• br0 is the 2 port 10G NIC connected to the switch
• br1 is the 4 port gigabit NIC used for internet access even without the switch essentially functioning like a home router with 4 ports)

This is the /etc/netplan/01-netcfg.yaml file:

 # This file describes the network interfaces available on your system
 # For more information, see netplan(5).
 network:
   version: 2
   renderer: networkd
   ethernets:
     enp0s8:  #WAN interface
       dhcp4: no
       dhcp6: no
        addresses: [xxx.xxx.xxx.xxx/24] #Static IP assigned by ISP
        gateway4: xxx.xxx.xxx.xxx       #Gateway Assigned by ISP
       nameservers:
        addresses: [xxx.xxx.xxx.xxx, 8.8.8.8] #DNS assigned by ISP
  optional: true
enp0s9:  #Internal gigabit NIC Interface not used
  dhcp4: no
  dhcp6: no
enp2s0f0:  #10G NIC Not used
  dhcp4: no
  dhcp6: no
enp2s0f1:  #10G NIC not used
  dhcp4: no
  dhcp6: no
enp5s0f0:  #port 1 NIC2 (10G NIC Bonded connected to switch)
  dhcp4: no
  dhcp6: no
enp5s0f1:  #port 2 NIC2 (10G NIC Bonded connected to switch)
  dhcp4: no
  dhcp6: no
enp8s0f0:  #port1 NIC3 (gigabit NIC acting as home router)
  dhcp4: no
  dhcp6: no
enp8s0f1:  #port2 NIC3 (gigabit NIC acting as home router)
  dhcp4: no
  dhcp6: no
enp9s0f0:  #port3 NIC3 (gigabit NIC acting as home router)
  dhcp4: no
  dhcp6: no
enp9s0f1:  #port3 NIC3 (gigabit NIC acting as home router)
  dhcp4: no
  dhcp6: no
bonds:
  bond0: #first 10G NIC, not used
   interfaces: [enp2s0f0, enp2s0f1]
   parameters:
     mode: 802.3ad
     lacp-rate: fast
     mii-monitor-interval: 100
  bond1: #second 10G NIC, bonded and connected to switch
   interfaces: [enp5s0f0, enp5s0f1]
   parameters:
     mode: 802.3ad
     lacp-rate: fast
     mii-monitor-interval: 100
bridges:
  br0: #this is the 10G nic connected to switch on 192.168.0.x subnet
   dhcp4: no
   dhcp6: no
   addresses: [192.168.0.1/24] #router IP address
   interfaces: [bond1]
   gateway4: 192.168.0.254
   routes:
   - to: 192.168.1.1 #router IP address (gateway/DNS) for the 4 port NIC subnet
   via: 192.168.0.1 #This allows the br1 to access the internet via the main router IP (located at 192.168.0.1), bypassing the switch
   parameters:
     stp: false
     forward-delay: 0
  br1: #home router 4 port gigabit bridge
   dhcp4: no
   dhcp6: no
   addresses: [192.168.1.1/24]
   gateway: 192.168.1.1
   interfaces: [enp8s0f0, enp8s0f1, enp9s0f0, enp9s0f1]
   parameters:
     stp: false
     forward-delay: 0

Note: a bridge was formed for bond 0 because I needed subnet 192.168.0.x on this NIC.

The br0 bridge allows the br1 bridge which is on subnet 192.168.1.x to connect to the internet via the router (192.168.0.1)

Next we need to configure DHCP on BOTH subnets. Here is are the iptables:

 # Allow LAN to access internet enp0s8 is WAN interface - mask the LAN IP with that of the WAN interface
 iptables -t nat -A POSTROUTING -o enp0s8 -j MASQUERADE

 #Allow all subnet access to the internet, The IP address is the WAN IP Address
 iptables -t nat -A POSTROUTING -o enp0s8 -j SNAT --to-source 69.59.44.75
 # Service rules

 # basic global accept rules - ICMP, loopback, traceroute, established all accepted
 iptables -A INPUT -s 127.0.0.0/8 -d 127.0.0.0/8 -i lo -j ACCEPT
 iptables -A INPUT -p icmp -j ACCEPT
 iptables -A INPUT -m state --state ESTABLISHED -j ACCEPT

 # enable traceroute rejections to get sent out
 iptables -A INPUT -p udp -m udp --dport 33434:33523 -j REJECT --reject-with icmp-port-unreachable

 # DNS - accept from LAN
 iptables -A INPUT -i br0 -p tcp --dport 53 -j ACCEPT
 iptables -A INPUT -i br0 -p udp --dport 53 -j ACCEPT
 iptables -A INPUT -i br1 -p tcp --dport 53 -j ACCEPT
 iptables -A INPUT -i br1 -p udp --dport 53 -j ACCEPT

 # SSH - accept from LAN
 iptables -A INPUT -i br0 -p tcp --dport 8880 -j ACCEPT
 iptables -A INPUT -i br1 -p tcp --dport 8880 -j ACCEPT

 # DHCP client requests - accept from LAN
 iptables -A INPUT -i br0 -p udp --dport 67:68 -j ACCEPT
 iptables -A INPUT -i br1 -p udp --dport 67:68 -j ACCEPT

 # drop all other inbound traffic
 iptables -A INPUT -j DROP

 # Forwarding rules
 # forward packets along established/related connections
 iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

 # forward from LAN (br0, br1) to WAN (enp0s8)
 iptables -A FORWARD -i br0 -o enp0s8 -j ACCEPT
 iptables -A FORWARD -i br1 -o enp0s8 -j ACCEPT

 # allow traffic from our NAT pinhole (PORT FORWARDING)
 iptables -A FORWARD -p tcp -d 192.168.0.100 --dport 8888 -j ACCEPT
 iptables -A FORWARD -p tcp -d 192.168.1.100 --dport 8888 -j ACCEPT

 # drop all other forwarded traffic
 iptables -A FORWARD -j DROP

The most important thing to note here is that whatever I have listed for br0, I have duplicated for br1

Lastly we need to setup the DHCP and ensure it works on both subnets. This is the /etc/default/isc-dhcp-server file:

 # On what interfaces should the DHCP server (dhcpd) serve DHCP requests?
 # Separate multiple interfaces with spaces, e.g. "eth0 eth1".
 INTERFACESv4="br0 br1"
 INTERFACESv6="br0 br1"

Lastly we need to modify the /etc/dhcp/dhcpd.conf file:

 .......
 subnet 192.168.0.0 netmask 255.255.255.0 {
    range 192.168.0.10 192.168.0.240;
    option routers 192.168.0.1;
    option domain-name "yourdomain.com";
    option domain-name-servers 192.168.0.1;
    option broadcast-address 192.168.0.255;
 }

 subnet 192.168.1.0 netmask 255.255.255.0 {
    range 192.168.1.10 192.168.1.240;
    option routers 192.168.1.1;
    option domain-name "yourdomain2.org";
    option domain-name-servers 192.168.1.1;
    option broadcast-address 192.168.1.255;
 }
 # Add a section to reserve static IPs --------  This must be done for the network switch to enable static routing if needed

 host DELL_PowerConnect_6248P {
         hardware ethernet XX:XX:XX:XX:XX:XX;
         fixed-address 192.168.0.254; #this is the layer 3 switch connected to NIC2 which serves as the gateway for the 192.168.0.x subnet
         option host-name "DELL_Powerconnect_6248P";  
 }

This worked for me. Now I can use the router like a home router even with the switch disconnected. PLEASE NOTE: This is by no means a list of what needs to be done to create your own ubuntu router. There are many other steps involved. This can easily be googled on the internet. Hopefully, this small tutorial helps someone in the future. Netplan is a pain, but rather straightforward. If an idiot like me can do it, anyone can. Happy Ubuntu-ing.

Status update. After about one hour, the WAN interface went down on the router. After some reading, it seems my above configuration was not quite correct. See the correction below. I have included comments (both for myself and others) in the code.

 ................
   bridges:
    br0:
     dhcp4: no
     dhcp6: no
     addresses: [192.168.0.1/24]
 #   gateway4: 192.168.0.254 # because we are using static routes, this option is not needed
     interfaces: [bond1]
     routes:
     - to: 0.0.0.0/0
     via: 192.168.0.254
     metric: 100
 #   Here, we have routed ALL subnets and all netmasks. Metric must be supplied if the gateway is not specified.
     - to: 192.168.1.1/24
     via: 192.168.0.1
     metric: 100
 #   Note: Even though we are specifying ALL subnets above, we can still have routing for a particular subnet so that all routes go to 192.168.0.254, but only 192.168.1.x subnet routes through 192.168.0.1
     parameters:
       stp: false
       forward-delay: 0   
    br1:
     dhcp4: no
     dhcp6: no
     addresses: [192.168.1.1/24]
     gateway: 192.168.1.1
 #   Here a gateway is specified, because there are no static routes
     interfaces: [enp8s0f0, enp8s0f1, enp9s0f0, enp9s0f1]
     parameters:
       stp: false
       forward-delay: 0

As you can see, I tweaked the config to prove a point. Even with ALL subnets being routed to 192.168.0.254 by default, the 192.168.1.x is an exception and is specifically routed to 192.168.0.1. This way I can still access the internet on the 4 port nic even if the switch is disconnected.

Now the router works and has not gone down for several hours. On to configuring bind, OpenVPN, and all the other good stuff.

I was able to learn all of this by looking at netplan examples specified at https://netplan.io/examples

Hopefully, this will prove of use with configuring netplan.