Home / server / Questions / 541573
Accepted
Alex
Asked: 2013-09-26 06:34:29 +0800 CST

Routing suddenly broke on a KVM guest

  • 772

I have a CentOS-based host and a KVM Debian-based virtual machine on it. A host has an Ethernet bridge on its external network interface, this bridge is used by KVM:

br0       Link encap:Ethernet  HWaddr 00:25:90:01:5E:92  
          inet addr:5.XX.XX.84  Bcast:5.XX.XX.255  Mask:255.255.255.0
          inet6 addr: fe80::fc54:ff:feaf:95b3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2893439068 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2943859744 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3109906781642 (2.8 TiB)  TX bytes:3271403241664 (2.9 TiB)

br0:0     Link encap:Ethernet  HWaddr 00:25:90:01:5E:92  
          inet addr:10.228.0.1  Bcast:10.228.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

The bridge has two IPs, one external and another one from a virtual LAN between the host and the guest. It acts as a default gateway for the guest. STP is turned off on the bridge.

The problem is that the guest got an odd routing rule somehow:

root@new:~# ip route get 50.31.164.148
50.31.164.148 via 5.XX.XX.81 dev eth0  src 10.228.0.250 
    cache  ipid 0x0dfb rtt 4.781s rttvar 4.297s ssthresh 7 cwnd 9
root@new:~#

5.XX.XX.81 is a default gateway of the host and I can't find this IP anywhere in static routing tables on the guest:

root@new:~# ip route list
default via 10.228.0.1 dev eth0 
10.116.0.0/16 via 10.116.0.146 dev tun0 
10.116.0.146 dev tun0  proto kernel  scope link  src 10.116.0.145 
10.228.0.0/24 dev eth0  proto kernel  scope link  src 10.228.0.250

I wonder how this was possible at all and what should we do to prevent situations like this? Of course, ip route flush cache saved us but we definitely want to eliminate problem itself not to blindly flush the routing cache periodically.

routing

1 Answers

  1. Best Answer

    (Converting my comment to an answer)

    ICMP redirects, see linux-ip.net/html/routing-icmp.html

    Your solution is quite unusual, normally one wouldn't bridge to a real interface but setup routing and masquerading on the host.

    • 1