Home / server / Questions / 569721
In Process
Ammar Lakis
Asked: 2014-01-25 12:09:57 +0800 CST

Can bonding 2 interfaces double the speed?

  • 772

I have 2 linux boxes running centos 6.5 each with 2 interfaces bonded together, linked to a Cisco 2960-S switch with lacp configured ports.

The configuration on the switch

port-channel load-balance src-dst-mac
!
interface Port-channel1
 switchport access vlan 100
 switchport mode access
!
interface Port-channel2
 switchport access vlan 100
 switchport mode access
!
interface FastEthernet0
 no ip address
!
interface GigabitEthernet0/1
 switchport access vlan 100
 switchport mode access
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 1 mode active
!
interface GigabitEthernet0/2
 switchport access vlan 100
 switchport mode access
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 1 mode active
!
interface GigabitEthernet0/3
 switchport access vlan 100
 switchport mode access
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 2 mode active
!
interface GigabitEthernet0/4
 switchport access vlan 100
 switchport mode access
 speed 1000
 duplex full
 spanning-tree portfast
 channel-protocol lacp
 channel-group 2 mode active
!

and on the both linux sides I've loaded the kernel bonding module with the configuration

alias bond0 bonding
options bond0 miimon=100 mode=4 lacp_rate=1

Now the problem is that I transfer many files from one server to another monitoring the traffic graphs showing that the speed doesn't exceed the 1Gb/s speed for the bonding interface bond0.

is there any problem with the configuration ? shouldn't the speed be doubled to 2Gb/s ?

bonding

4 Answers

  1. LACP will not split packets across multiple interfaces for a single stream/thread. For example a single TCP stream will always send/receive packets on the same NIC.

    See the following post for reference:

    Link aggregation (LACP/802.3ad) max throughput

    Hope this helps.

    • 12

  2. The way Link Aggregation works is by using a hashing algorithm to decide which packets should go out which port.

    Packets from the same source MAC address, to the same destination MAC address, will always go out the same port.

    Some Link Aggregation implementations support using layer 3 (IP addresses) and even layer 4 (TCP/UDP Port number) as part of the hash, but this is not that common.

    This is why you only get 1 gbit/sec when transferring files from 1 server to another.

    If the OS and Switch will both support layer 3, you can get more speed by using multiple IP addresses. However because of the way the hashing algorithm works, there is a 50/50 chance that both streams will end up going out the same link.

    • 7

  3. My understanding of network bonding is that you cannot exceed the link speed of the member interfaces in one connection. A connection will stick to one interface in the bond after it is established.

    However, connections are now split between the two interfaces. If you were to have two connections running from server A to server B, then the connections shouldn't start bottle-necking each other as far as bandwidth goes because they will be traveling across different interfaces. Your total bandwidth using multiple connections should be 2Gb/s, but each connection will be limited to a maximum of 1Gb/s.

    • 4

  4. Its also worth considering the hash algorithm being used by Linux. Some versions of the bond driver use very simplistic hash algorithms e.g l2 hash on linux 3.6.5 is just the xor of the last byte of the source and destination MAC which leads to unbalanced traffic in a lot of circumstances. Changing the hash algorithm to l2+l3 will help a lot.

    • 1