Home / server / Questions / 406672
Accepted
Bryan Agee
Asked: 2012-07-12 00:53:55 +0800 CST

Link bonding across multiple switches?

  • 772

I've read up a little bit on bonding nics with ifenslave; what I'm having trouble understanding is whether there is special configuration needed in order to split the bonds across two switches. For example, if I have several servers that all have two nics each, and two separate switches, do I just configure the bonds and plug 1 nic from each into switch #1 and the other from each into switch #2? or is there more to it than that?

If the bonds are active-backup, will a nic failure on single machine mean that server may become disconnected since the rest of the machines are using the primary nic and it's using the secondary? Or do you link the switches with one cable as well?

networking switch bonding failover lag

4 Answers

  1. Best Answer

    From the Linux Kernel Network Bonding documentation:

    In a topology such as the example above, the active-backup and broadcast modes are the only useful bonding modes when optimizing for availability; the other modes require all links to terminate on the same peer for them to behave rationally.

    Note, the example referenced in the quote is for a single host with 2 (or more) NICs, with each NIC connected to a separate switch. The switches have a link between them (ISL or Inter-Switch Link) connecting them together.

    • 6

  2. It depends on the bonding mode which you're using. The 802.3ad mode requires that the switch(es) support 802.3ad and are specifically configured for this. The other modes (balance-rr, active-backup, balance-xor, balance-tlb and balance-alb) don't require specific switch support.

    I'm not sure how the inter-switch connectivity affects the choice of bonding mode, but I suspect that the switches need to be connected to each other and have STP enabled. This page looks like a good resource for bonding.

    • 5

  3. ... what I'm having trouble understanding is whether there is special configuration needed in order to split the bonds across two switches.

    If you want to bond a link across switches, first you need a switch that supports it. LACP (802.3ad) only supports bonding within a single switch. All the inter-switch ethernet bonding protocols I know of are proprietary.

    Contact your switch vendor to understand whether they have a multi-chassis ethernet bonding protocol that works with your server OS.

    EDIT:

    as mgorven pointed out in the comments, you can bond across multiple switch without needing 802.3ad

    • 3

  4. The only bonding mode that achieves the high-availability we're looking for is the active-backup bonding, with one cable from each bonding group attached to diverse switches, which themselves are interconnected. This allows for any given switch to fail, pushing all of the traffic to the other switches and NICs connected thereto; or any number of individual NICs can fail provided that no two are in the same bond and all switches are operational.

    802.3ad requires managed switches with support for it, and xor- or rr-balanced require that the switches in question support port grouping of some kind.

    • 2