Home / server / Questions / 1014447
In Process
Basj
Asked: 2020-04-28 10:24:02 +0800 CST

How to do this load-balacing properly?

  • 772

Let's say I rent 6 dedicated servers by a server provider for my website http://example.com:

#1: 203.2.113.1
#2: 203.2.113.2
#3: 203.2.113.3
...
#6: 203.2.113.6

each of them having a maximum upload rate of 10 MB/s.

I decide to use machine #1 as load-balancer, so by my registrar, I set up a DNS "A" record: example.com A 203.2.113.1.

Two cases:

  • 50 clients want to receive a video stream @200KB/s from my service at the same time: everything seems fine:

    They connect http://example.com in their browser, the DNS gives them IP 203.2.113.1, their browser makes a HTTP request to this IP, the load-balancer (machine #1) receives the HTTP request, gives the task to machine #2, #3, ..., #6, and these machines send the stream to the client by passing the data first through the load balancer. Am I correct that the client wouldn't accept a response from 203.2.113.2 because their request was sent to 203.2.113.1?

  • 200 clients want to receive a video stream @200KB/s from my service: it does not work:

    The clients connect http://example.com in their browser, the DNS gives them IP 203.2.113.1, their browser makes a HTTP request to this IP, the load-balancer (machine #1) receives the HTTP request, gives the task to machine #2, #3, ..., #6. Each of these 5 machines process 1/5th of the 200 clients, i.e. 40 clients * 200KB/s = 8 MB/s.

    If #2, #3, ..., #6 could send the data directly to the clients, it would be fine (8 MB/s).

    But since the data has to transit via the load-balancer machine #1, it does not work because machine #1 cannot upload at 40 MB/s.

How to configure the load-balancer (machine #1) to solve this problem?

Or would it be possible that the load-balancer passes the requests to machines #2, #3, ..., #6 and that then, they answer to the clients directly, without going through machine #1 for the responses?

domain-name-system networking upload load-balancing bandwidth

4 Answers

  1. A single machine running as a load balancer is a bottleneck. You should look at Round Robin DNS (second link) aka DNS Load Balancing. It's not super reliable, your machines won't be evenly loaded, but each machine should get some load.

    You'll want to test this to make sure it works, using a distributed load testing service. A key is the client regularly looking up DNS for your domain.

    You should also look at stream caching. CloudFlare offers the service, as I expect others do, but I haven't used it so don't know much about it.

    • 1

  2. Are you sure that something like http://www.haproxy.org/ would not be able to handle traffic from 200+ customers? Read this for example: http://www.haproxy.org/#perf .

    Effectively all it has to do is route TCP/IP traffic to a specific machine and back. CPU-wise that's not very expensive except truly huge number of connections. I'd be more worried about the capacity of the network controller and you'd also have to remember about things like maximum number of open files etc that can negatively impact handling network traffic. Seems like careful tuning of the network stack is more important than capacity of HAProxy:

    All these micro-optimizations result in very low CPU usage even on moderate loads. And even at very high loads, when the CPU is saturated, it is quite common to note figures like 5% user and 95% system, which means that the HAProxy process consumes about 20 times less than its system counterpart. This explains why the tuning of the Operating System is very important.

    • 1

  3. Or would it be possible that the load-balancer passes the requests to machines #2, #3, ..., #6 and that then, they answer to the clients directly, without going through machine #1 for the responses?

    Yes, it's called "Direct Return" load balancing, and any decent L4 load balancer will be able to do it.

    • 1

  4. I just found potential solutions to this problem, here is how it goes:

    Solution 1

    • A client's browser connects to example.com, i.e. after DNS resolution, to server #1 (203.2.113.1)

    • Server #1 looks which of the machines #2, #3, ..., #6 is available, let's say #5 has the smallest load currently and can take the job

    • Server #1 sends the HTTP response to the client : yourworker=203.2.113.5 or yourworker=worker5.example.com

    • The client receives this, and knows it now has to connect 203.2.113.5 or worker5.example.com (via AJAX/XMLHttpRequest) to receive the video stream.

    Thus the video traffic never goes through load-balancer machine #1 which just sends a few bytes (yourworker=...) for each client at the beginning, i.e. no more bottleneck.

    The only potential problem is: if the browser has loaded the page http://example.com from 203.2.113.1, can it do AJAX/XHR to 203.2.113.5 or worker5.example.com to receive the video stream, or will this be declined as CORS?

    Edit: Tested, working. It works if you enable Access-Control-Allow-Origin, example with Apache .htaccess (we can put a more precise control, but here it's ok for the test):

    Header set Access-Control-Allow-Origin "*"

    Solution 2

    Another simpler solution: can't the load balancer #1 just analyze which machine #2, #3, ..., #6 is available, and send this in return to the client:

    <?php
$i = which_worker_available();
header('Location: https://worker' . $i . '.example.com/');
?>

    ?

    Then all the rest of the traffic will never go through the load-balancer again. No more bottleneck.

    Problem here: the URL displayed in the client's browser will change from https://example.com to https://worker5.example.com which is not very nice for the end user.

    Edit: Tested, working (but indeed the URL in the browser bar changes from http://example.com to http://worker5.example.com/...)

    • 0