Home / server / Questions / 524054
Accepted
tfrederick74656
Asked: 2013-07-18 06:57:09 +0800 CST

Simple Multihomed Linux Server Issue

  • 772

I have a Linux server running CentOS 6.4 that is used as an iSCSI target. The server is multi-homed with two NICs, both on the same subnet. iSCSI multipathing takes care of the load-balancing/failover, so all I need is for each NIC to operate independently. How do I configure this system to avoid all the weird routing and ARP issues that always come along with a multi-homed setup, such as traffic being returned on a different interface from the source, or one interface accepting traffic sent to the IP of the other. The bonding solution is not an option, as it has issues with the iSCSI connection.

The important stuff:

  • eth0: IP 10.1.1.242 / SN 255.255.252.0 / GW 10.1.1.254
  • eth1: IP 10.1.1.243 / SN 255.255.252.0 / GW 10.1.1.254
linux

3 Answers

  1. Best Answer

    I realized I never followed up to this question. Using some excellent internet resources here and here, I came up with the following configuration. Hopefully this helps someone out.

    Assume you have two interface, eth0 and eth1, with IP addresses of 10.1.1.242 and 10.1.1.243. This is all on a /22 network with a default gateway of 10.1.1.254.

    • First create two routing tables, one for each NIC:

      echo "1 lan1" >> /etc/iproute2/rt_tables`
echo "2 lan2" >> /etc/iproute2/rt_tables`

    • Next, add routes for each interface to the appropriate tables:

      ip route add 10.1.0.0/22 dev eth0 src 10.1.1.242 table lan1
ip route add default via 10.1.1.254 dev eth0 table lan1
ip route add 10.1.0.0/22 dev eth1 src 10.1.1.243 table lan2
ip route add default via 10.1.1.254 dev eth1 table lan2

    • Finally, add rules to determine which table is used:

      ip rule add from 10.1.1.242/32 table lan1
ip rule add to 10.1.1.242/32 table lan1
ip rule add from 10.1.1.243/32 table lan2
ip rule add to 10.1.1.243/32 table lan2

    This should keep traffic from crossing NICs internally, allowing you to preserve redundancy or use each NIC for different functionality.

    • 6

  2. You're looking for ip route and doing policy routing, each interface will have it's own routing table, and make routing decisions (including which interface to send traffic on, and from which address) based on those tables.

    This is an excellent site explaining the concepts from the beginning, but you can jump right into policy routing if you'd like..

    • 0

  3. You will need to look at the arp_announce and arp_ignore variables to make certain that the adapters respond as you would expect.

    I'd suggest reviewing the documentation here. The values in that doc for VIPs would probably be appropriate.

    In /etc/sysctl.conf:

    net.ipv4.conf.eth0.arp_ignore = 1
net.ipv4.conf.eth0.arp_announce = 2
    • 0