We have some windows servers (2008 R2) and we are experiencing a strange network issue on them. The best way that I can describe it is that we are experiencing low throughput (but high bandwidth) only when operating over internet latency (20ms or more) networks.
Symptoms:
1.) The servers in question are located in two tier 1 datacenters with great, wide internet pipes.
2.) If we RDP onto servers in Data Center #1, we can run SpeedTest.NET test and get hundreds of meg down/up
3.) If we download a CentOS ISO image (basically a big file) on a server, we can get between 6 - 20 Megabytes/sec bandwidth
4.) If we connect to a web server we run on a FreeBSD 9.0 server (also in the data center) from another datacenter (so, freebsd in dc #1, windows server in dc #2) and download a big file (100s of MB) we also get great performance. (again, many megabytes/sec)
5.) If we connect from a server in DC #1 to another windows server in DC #1 we get awesome performance (nearly 1gb/sec).
6.) If we connect from a windows server box in DC#1 to a windows server box in DC#2 we get ~450 kilobytes/sec (really poor throughput). but we can open multiple browsers/windows and download as many times as we want (simultaneously) and still get exactly the same 450 kb/sec.
- We ran NTTTCP and a windows port of netperf between windows servers in DC#1 and DC#2 and, in every case we can find, we get the same throughput (~450 kb/sec) on each download.
8.) We ran a whole bunch of other tests that allow us to conclusively eliminate firewalls etc. as the source of our issue (also, see test 4, as the FreeBSD server is communicating though the same [hardware] stack as the windows servers...)
We tried disabling the TCP window scaling on a pair of windows servers to no effect.
So ServerFault community: it seems like it must be a windows TCP setting of some sort -- what causes this sort of issue?
EDIT
1.) We installed VMWare/CentOS on one of the windows server 2008R2 boxes having the issue, with bridged networking, guess what great throughput! So, VMWare image, on windows host, greate performance, go figure!
Try disabling the TCP Chimney and autotuning: