when i set up adapter teaming on an HP server, one of the options i'd like to enable is round-robin for transmit load balancing because i need the full teamed throughput to a single host. (this is a live migration network for hyper-v.)
when i pick that option, i get a message saying "Packet order not guaranteed". for my use at least, i don't see a problem since my traffic is TCP. but i'm interested in knowing any general problems with out of order packets.
so, what are the potential problems with out of order packets?
Fibre-Channel-over-Ethernet requires, like 'regular' Fibre-Channel in-order-delivery, hence why FCoE required DataCentreEthernet (DCE) which guarantees in-order-delivery. But you're right that there's few reasons in a purely TCP environment to require in-order.
They have sequence numbers for a reason, packets are dropped all the time and retransmitted. If you're using round-robin transmit load balancing then it means its going to send packet A down the first link, packet B down the second. Now... if these went to the same physical switch, it would probably never misbehave, but think of a scenario where each link went to the same VLAN on different switches and the trunk between those switches is inadequate. If the destination MAC address is on the one switch then everything will have to cross that link and you're going to have different latencies within a stream.
See how this gets fun? For transmit load balancing, you should be sure either your switches are stacked, or your fabric/interswitch links are properly sized, or if you're going to the same physical switch, you're groovy already :)
Hope that helps.