Home / server / Questions / 316623
In Process
Arlukin
Asked: 2011-09-29 15:38:52 +0800 CST

What is the correct way to setup a bonded bridge on Centos 6 for KVM guests?

  • 772

What is the correct way to setup a bonded bridge on Centos 6 for KVM guests?

I'd currently playing around with a setup of two KVM-hosts which will each host several guests. I have two HP DL380:s with 4 nics each. I'd like to use two nics (eth0, eth1) in a active-backup (mode=1) bond, for failover reasons, facing internet. Then i'd like to have the two other nics (eth2, eth3) also in a active-backup bond, facing a admin/back net.

On top of the bond I need a bridge that the KVM guests will use to access front or back network.

On the interwebz I have found many different ways to configure this. Some are just mentioning bonding, some just bridging and some are tring to combine it. None that I have found has metioned what will happen if I use a front and back net with many hosts.

Some of my problems/questions are.

  • I got a fetich on correct config files, the way the developers thought they should be, not just working config files.
  • I got error "kernel: bond0: received packet with own address as source address". Both for bond0 and bond1.
  • Will the traffic automtically be forwarded from the back-net to the front-net. Should I use ebtables/iptables or something to disable the forwarding traffic?
  • Do I need to use Spanning Tree Protocol (STP)?
  • Do I need any specific routes?

Here is a nice picture how the environment looks (at least a part of it.)

Network schema

Here are my relevant config files.

/etc/sysconfig/network

NETWORKING=yes
HOSTNAME=host1
GATEWAYDEV=br0
NETWORKING_IPV6=no

/etc/sysconfig/network-scripts/ifcfg-eth0 -- ifcfg-eth3

DEVICE="ethX"
NM_CONTROLLED="no"
ONBOOT=yes
HWADDR=xx:xx:xx:xx:xx:xx
SLAVE=yes
MASTER=bondX
HOTPLUG=no
BOOTPROTO=none

/etc/sysconfig/network-scripts/ifcfg-bond0 -- ifcfg-bond1

DEVICE=bondX
BONDING_OPTS="miimon=100 mode=1"
ONPARENT=yes
BOOTPROTO=none
BRIDGE=brX

/etc/sysconfig/network-scripts/ifcfg-br0

DEVICE=br0
TYPE=Bridge
ONBOOT=yes
DELAY=0
BOOTPROTO=none

/etc/sysconfig/network-scripts/ifcfg-br1

DEVICE=br1
TYPE=Bridge
ONBOOT=yes
DELAY=0
BOOTPROTO=static
IPADDR=10.0.1.100
NETMASK=255.255.255.0

Update 1

  • Added /etc/sysctl.conf
  • Removed ip from ifcfg-br0. The host shouldn't be accessible from internet, only from admin net.

* Update 2*

  • Removed changes to /etc/sysctl.conf. Don't need to enable iptables.
centos kvm-virtualization bonding bridge

2 Answers

  1. Not sure about CentOS 6, but on Fedora the bonding module is not added to the Linux kernel by default and therefore you need to create a file /etc/modprobe.d/bonding.conf with content

    alias bond0 bonding

    Reboot, and you should see bonding module loaded during boot.

    Since you have two bonded interfaces you might have to add another alias line for bond1 as well. However I have never tried that.

    Suggest you get one working and then worry about setting up the second.

    Other issue you raised about the bridge, other points to note, these configurations work with the network daemon but I don't believe they work with NetworkManager. Are you running the network or the NetworkManager daemon?

    And finally, there are different ways to configure netfilter to handle bridged interfaces. At least on Fedora 12+, the default is to disable netfilter on bridges. However you can change this by editing /etc/sysctl.conf and setting 

    net.bridge.bridge-nf-call-iptables=1

    Do the same for arp and ipv6 and in this file you also need to set

    net.ipv4.ip_forward=1

    Flush your FORWARD chain and replace with iptables rule

    iptables -A FORWARD -m physdev --physdev-is-bridged -j ACCEPT
    • 2
    1. forwarding to iptables will work, but what for? this will definitely create extra overhead
    2. the networking schema seems right, though I'd try and get the hardware to use mode 4 instead of 1 to be on the safe side, and to also aggregate the traffic
    3. not having an IP addres on br0 and br1, do you have other NICs you can access the host by? You really don't want to lock yourself out (unless you also have an iLO of course)
    4. the kernel message you posted is usually related to using bonding modes incompatible with bridging (0 and 6). Can you make sure youre actually at mode-1 (/proc/net/bonding/...) usually contains an indication
    5. forwarding between subnets should not happen without specific configuration, however, the hosts' kernel should be able to determine which packet is destined where, in br0 and br1 are on different subnets, so a VM on br0 should be able to talk on the br0 network, and if you want it to also be able to communicate on br1, add another virtual NIC and attach it there
    • 2