Home / server / Questions / 397741
Accepted
Trey
Asked: 2012-06-12 13:53:12 +0800 CST

FreeBSD slow transfers - RFC 1323 scaling issue?

  • 772

I think I may be having an issue with window scaling (RFC 1323) and am hoping that someone can enlighten me on what's going on.

  • Server: FreeBSD 9, apache22, serving a static 100MB zip file. 192.168.18.30
  • Client: Mac OS X 10.6, Firefox 192.168.17.47
  • Network: Only a switch between them - the subnet is 192.168.16/22 (In this test, I also have dummynet filtering simulating an 80ms ping time on all IP traffic. I've seen nearly identical traces with a "real" setup, with real internet traffic/latency also)

Questions:

  • Does this look normal?
  • Is packet #2 specifying a window size of 65535 and a scale of 512?
  • Is packet #5 then shrinking the window size so it can use the 512 scale and still keep the overall calculated window size near 64K?
  • Why is the window scale so high?

Here are the first 6 packets from wireshark. For packets 5 and 6 I've included the details showing the window size and scaling factor being used for the data transfer.

No. Time Source Destination Protocol Length Info

108 6.699922 192.168.17.47 192.168.18.30 TCP 78 49190 > http [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=8 TSval=945617489 TSecr=0 SACK_PERM=1

115 6.781971 192.168.18.30 192.168.17.47 TCP 74 http > 49190 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=512 SACK_PERM=1 TSval=2617517338 TSecr=945617489

116 6.782218 192.168.17.47 192.168.18.30 TCP 66 49190 > http [ACK] Seq=1 Ack=1 Win=524280 Len=0 TSval=945617490 TSecr=2617517338

117 6.782220 192.168.17.47 192.168.18.30 HTTP 490 GET /utils/speedtest/large.file.zip HTTP/1.1

118 6.867070 192.168.18.30 192.168.17.47 TCP 375 [TCP segment of a reassembled PDU]

Details:

Transmission Control Protocol, Src Port: http (80), Dst Port: 49190 (49190), Seq: 1, Ack: 425, Len: 309
Source port: http (80)
Destination port: 49190 (49190)
[Stream index: 4]
Sequence number: 1 (relative sequence number)
[Next sequence number: 310 (relative sequence number)]
Acknowledgement number: 425 (relative ack number)
Header length: 32 bytes
Flags: 0x018 (PSH, ACK)
Window size value: 130
[Calculated window size: 66560]
[Window size scaling factor: 512]
Checksum: 0xd182 [validation disabled]
Options: (12 bytes)
No-Operation (NOP)
No-Operation (NOP)
Timestamps: TSval 2617517423, TSecr 945617490
[SEQ/ACK analysis]
TCP segment data (309 bytes)
freebsd tcp rfc tcp-window-scaling

2 Answers

  1. 1.) 512 isn't really a high window scale - it's just saying to shift the offered window size to the left by 9 bits. Setting the window size to 130 (otherwise a very, very low value) and then applying a scale factor of 512 gets you to 66560 (130<<9).

    2.) 100M is likely too small a file. The fact that the scale has been negotiated at all suggests that things are working OK. Try a larger file to better observe the behavior. If nothing else you'll get a better sense of the real throughput.

    3.) Also bear in mind that the behavior of particular clients can actually specifically override the behavior of the OS - the built in FTP client in Solaris, for example, used to limit window size to 64K regardless of how the OS was configured.

    • 1
  2. Best Answer

    I got word from the sysadmin team that this issue was being caused by some issue with the VMWare net driver not respecting / playing nice the sysctl tunables. Throughput with the same setup on physical hardware has throughput with a reasonable percentage for the pipe rather than the 1/10th or less we saw with VMware.

    • 0