Home / server / Questions / 225185
Accepted
Marnix van Valen
Asked: 2011-01-22 07:31:34 +0800 CST

Route return traffic to correct gateway depending on service

  • 772

On my office network I have two internet connections and one CentOS server running a website (HTTPS on port 443). The website should be publicly accessible through the public IP of the first internet connection (ISP-1). The other internet connection, ISP-2, id the default gateway on the network. Both internet connections have routers (the household-kind) with NAT, SPI firewalls etc. The router on ISP-2 is a Netgear WNDR3700 (aka N600) with original firmware.

The problem is that the website is unreachable. Looks like incoming traffic on ISP-1 will reach the server but the returning traffic is routed through ISP-2, effectively making the site unreachable. As far as I can tell I can't do port based routing on the WNDR3700.

What are my options to make this work? I've been looking at implementing an iptables / routing based solution on the server itself but haven't been able to make that work.

Update: Note that the server has one network interface connecting it to both routers.

linux routing iptables

5 Answers

  1. I had the same issue, but I solved only with iproute2 (source routing). Marking with iptables wasn't necessary:

    echo "101 webtraffic" >> /etc/iproute2/rt_tables
ip route add default table webtraffic via $ISP1_GW_LAN_IP
ip rule add from $ISP1_IP table webtraffic

    The point is that not only web may use the ISP1 connection. You may choose. It's good because you may connect to the server from ssh from both connections if someone fails. As a CentOS user I created the following files so my changes weren't lost after reboot:

    echo "default table webtraffic via $ISP1_GW_LAN_IP" >> /etc/sysconfig/network-scripts/route-eth1
echo "from $ISP1_IP table webtraffic" >> /etc/sysconfig/network-scripts/rule-eth1
    • 5
  2. Best Answer

    If I am understanding your intentions correctly, you want your webserver to normally use ISP-2 as its default gateway for outgoing traffic, with the exception of its responses to external web requests, which must transit via ISP-1 instead. Here is a sketch of a solution using policy routing:

    echo "101 webtraffic" >> /etc/iproute2/rt_tables

ip route add default table webtraffic via $ISP1_GW_LAN_IP
ip rule add fwmark 1 table webtraffic
iptables -t mangle -A OUTPUT -d \! $LAN_NET_PREFIX \
                             -p tcp -m tcp --sport 443 \
                             -j MARK --set-mark 1

    where:

    • LAN_NET_PREFIX is your LAN's network prefix (e.g. 192.168.100.0/24), and
    • ISP1_GW_LAN_IP is the LAN IP address of your gateway to ISP-1 (e.g. 192.168.100.100).

    The first ip command sets the default route on the webtraffic table to your ISP-1 gateway, and the second ensures that packets marked 1 are routed using the webtraffic table. Finally, the iptables rule marks the appropriate outgoing packets, ensuring that their next hop will be towards ISP-1.

    Here is an alternate solution that uses an experimental iptables module, the ROUTE target:

    iptables -t mangle -A POSTROUTING -d \! $LAN_NET_PREFIX \
                                  -p tcp -m tcp --sport 443 \
                                  -j ROUTE --gw $ISP1_GW_LAN_IP

    This rule would override the routing decision for outgoing web response packets, sending them to your ISP-1 gateway, instead of to the default ISP-2. All other traffic, including web responses to clients on your LAN, would not be affected. As has been pointed out in the comments, the ROUTE target is very likely not to be implemented on any system that has not explicitly patched it into the kernel, since it is experimental.

    • 3

  3. This is called asynchronous routing. You have to point the default gateway of the webserver to the IP address of the router on ISP-1. If the clients that are accessing the web server came from the same IP you can route this without changing the default gateway, or, you can implement NAT on the ISP1 gateway to act like a reverse proxy and then route it at the web server. Regards.

    • 0

  4. Firewall people do inside nat to get over this problem. The inbound connection is natted to the firewall inside interface address and therefore does not have a routing problem at the server.

    If the both inbound connections end up in same interface in same IP you have very limited (none?) options. See if you can get another IP for the same computer/interface and have one inbound connections in one IP and anoter inbound in another IP. After this you can do source routing easily.

    If you can't get different sources connect to different IP addresses you can use iptables mac to redirect packets depending on MAC they were coming from. 

       [!] --mac-source address
          Match  source  MAC  address.    It   must   be   of   the   form
          XX:XX:XX:XX:XX:XX.   Note that this only makes sense for packets
          coming from an Ethernet device and entering the PREROUTING, FOR‐
          WARD or INPUT chains.

    Hope this helps.

    • 0

  5. Assuming your Webserver is on the same subnet as your workstations, why not just setup split DNS so that on the inside, yourwebsite.com resolves to it's internal IP address? Sure beats complicated asynchronous routing.

    Also, investing a router than can support multiple WAN connections would make life a lot easier for you (and allow you do load balancing/failover between both Internet connections).

    • 0