Home / server / Questions / 910948
In Process
gijs007
Asked: 2018-05-07 04:21:09 +0800 CST

Improve single HTTP transfer bandwith over high latency connection (Ubuntu Linux)

  • 772

I'm trying to improve the file transfer speed for my web server in Roubaix, France (hosted by OVH) over high distance/latency connections.

I know that TCP connections are affected by higher latency, however I've heard that tuning the TCP stack and using better congestion protocols can make a huge difference. I've seen whitepapers about TCP BBR pushing 10GB/s over high latency connections.

I'm expecting to push the connection near to the 1Gb/s limit, as I can with low latency (<20ms RTT) connections.

The server setup

  • OS: Ubuntu 18.04
  • Web server: Apache 2.4.29
  • Connection: 1Gb/s

Test file: Ubuntu 17.10.1 ISO (1.39GB)

Test "clients":

  • OS: Windows Server 2016
  • Connection: 1Gb/s

Locations:

  • New York, USA (73ms RTT)
  • Sydney, Australia (also OVH network, 267ms RTT)

Results:

New York has a download speed of around 3.2 - 3.4 MB/s

Sydney has a download speed of around 900-950KB/s

What's interesting is that using a download manager, to create more HTTP/TCP connections causes an increases in the download speed. Meaning the link is not saturated.

Tested tweaks:

First I've tested with several TCP congestion protocols:

  • TCP Cubic (default)
  • TCP BBR
  • TCP Hybla
  • H-TCP

Changes in the congestion protocol make almost no difference, at most 200KB/s on the New York connection and <50KB/s on the Sydney connection.

I've also tried setting other kernel parameters, such as the ones suggested here: https://medium.com/@frjalex/linux-kernel-high-performance-networking-how-to-2d35f5900974

These settings have no observable effect on the download speed.

Question

How can I tune the Linux TCP/IP stack, so that single TCP (HTTP) connections max out the available bandwidth on high latency connections?

Issue

After running a test with wget on Linux I found that the issue was with the testing clients.. Apparently Windows (server 2016) does not increase the TCP receive window enough to maximize the connection..

networking

1 Answers

  1. After running a test with wget on Linux I found that the issue was with the testing clients.. Apparently Windows (server 2016) does not increase the TCP receive window enough to maximize the connection..

    • 0