Home / server / Questions / 607446
Accepted
Evgeniy Berezovsky
Asked: 2014-06-24 22:42:55 +0800 CST

tcpreplay throughput on eth0 well below (iperf confirmed) max

  • 772

I'm trying to replay captured UDP multicast packets at a rate of a couple of hundreds of Mbps, but it maxes out at about 86 Mbps. (For some strange reason, at other times during the day it consistently maxes out at one tenth of that rate, about 8.8 Mbps)

tcpreplay on eth0

Yields only about 86Mbps:

> tcpreplay -ieth0 --topspeed x.pcap 
sending out eth0 
processing file: x.pcap
Actual: 459424 packets (137973257 bytes) sent in 12.23 seconds.         
Rated: 11281542.0 bps, 86.07 Mbps, 37565.33 pps

iperf on eth0

iperf on the other hand goes up to about 800Mbps. Note that this is true even during those times when tcpreplay only yields about 8.x Mbps)

iperf -c 192.168.20.55 -u -T 1 -t 5 -i 1 -b10000M
------------------------------------------------------------
Client connecting to 192.168.20.55, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 192.168.20.55 port 50238 connected with 192.168.20.55 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec  96.9 MBytes   813 Mbits/sec
[  3]  1.0- 2.0 sec  96.9 MBytes   813 Mbits/sec
[  3]  2.0- 3.0 sec  97.8 MBytes   820 Mbits/sec
[  3]  3.0- 4.0 sec  96.4 MBytes   809 Mbits/sec
[  3]  4.0- 5.0 sec  96.4 MBytes   809 Mbits/sec
[  3]  0.0- 5.0 sec   484 MBytes   812 Mbits/sec
[  3] Sent 345538 datagrams
[  3] Server Report:
[  3]  0.0- 5.0 sec   400 MBytes   671 Mbits/sec   0.071 ms 60212/345537 (17%)
[  3]  0.0- 5.0 sec  1 datagrams received out-of-order

tcpreplay on lo

Just to make sure tcpreplay isn't waiting on my disk:

> tcpreplay -ilo --topspeed x.pcap 
Warning in sendpacket.c:sendpacket_open_pf() line 669:
Unsupported physical layer type 0x0304 on lo.  Maybe it works, maybe it wont.
    See tickets #123/318
sending out lo 
processing file: x.pcap
Actual: 459424 packets (137973257 bytes) sent in 0.56 seconds.
Rated: 246380816.0 bps, 1879.74 Mbps, 820400.00 pps

What explains tcpreplay being slow on eth0 and is there a way to improve on that?

udp

2 Answers

  1. What version of tcpreplay? Newer versions are faster then older versions. You really want to run the latest 4.x which supports injection via NETMAP.

    Also, are you sure you're on a 1Gbps or better interface? Are you connected to a 100Mbps switch port or for some reason negotiated 100Mbps? Look in /var/log/messages or dmesg to see how your interface negotiated.

    • 1
  2. Best Answer

    The problem seems to be caused neither by my box, nor by tcpreplay. Testing on a different machine, using the following setup

                    +-----------+
                | L3 switch |
                +-+----+----+
                  |    |
       +----------+    |
       |               |
 +-----+-----+         |
 | L2 switch |         |
 +-----+-----+         |
       | Cable 1       | Cable 2
   +------------------------+
   |  NIC 1          NIC 2  |
   |                        |
   |         My PC          |
   |                        |
   +------------------------+

    I get these figures

    Cable     Mbs        pps 
--------------------------
     1    8.35    6174.75 
     2  601.07  444444.44

    I also switched cables to make sure it's not the NIC that is slow, and it isn't. My guess: It's the L2 switch's fault.

    • 0