Home / server / Questions / 356038
Accepted
qryss
Asked: 2012-02-02 18:20:07 +0800 CST

What tools should I use to assess bandwidth for web server farm..?

  • 772

I'm contemplating moving datacentre and I need to know the size of pipe I'll need in order to get a quote.

Currently I'm charged on GBs of traffic per month (currently about 42GB inbound - so requests only) but the new mob would rent me a pipe instead and the size of it determines the price.

I have a number of Centos web servers and one database server behind a loadbalancer; the webservers access the database server but the public do not (directly, that is).

The obvious place to measure bandwidth utilisation is at the loadbalancer but I don't have access to it and my current datacentre want an arm and a leg to monitor it for me.

Is this something I can easily do from my Linux web servers?

I've started to look at tools like ntop and bandwidthd but thought I'd seek some expert advice from here first.

I guess what I need to do is look at traffic between each webserver and the IP of the loadbalancer and add them together. Tools like ntop show traffic between the servers and the remote IP but not between the server and an intermediary IP...

Any clues anyone?

network-monitoring bandwidth network-speed network-traffic measurement

4 Answers

  1. Best Answer

    Do you really need an exact figure (whatever that means) to do this? Is knowing you currently use 100Mbps or 150.3528Mbps going to make a difference? Maybe you don't need to gather this data to get prices.

    What pipe/price options do they offer? If they don't offer these, ask for quotes in bands that make sense to you. You may get a better deal by not revealing up front how much you actually need, and instead get them to reveal their pricing structure. And the info may be useful in future when you need to upgrade.

    What does paying what you can afford get you? If it gets you way more than you expect to need now or in the next few months, then go down a level and ask again if you need that much pipe. Repeat till you are looking at a good price/pipe balance.

    Then look at the options to upgrade. How long will it take to upgrade? Are their additional costs to change later? How much will it cost you to have too small a pipe for the time it takes to realise and implement change? That might direct you to go for a bigger pipe.

    Answers to those questions will also be helpful in the happy circumstances of needing extra bandwidth because you are so successful in the future!

    • 1

  2. Inbound web traffic? Do you mean from the perspective of a browser or your server?

    Your weblogs (hopefully apache?) should have the size of the content delivered out to users.

    You can use those byte counts with timestamps to determine how much of your traffic is going out.

    Be sure to convert from Bytes/s to Bits/s (8 bits in a byte).

    Often pipes are billed at 95th percentile. (your top 5% of data is thrown out)

    Some napkin math: 42GB/Month =~ 1.4GB/day =~ 0.02MB/s =~ .13Mb/s That's average. Your traffic peaks are likely 2-3X that. So a 1Mbps link might do the trick.

    It depends on your traffic patterns.

    As I said, you can get that via apache logs.

    • 0

  3. you could run something like iptraf or just or reset your ifconfig counters. on each of your servers, take a sample over 24 hours to find out some average numbers. that will give you a ball park figure.

    Have you thought about a CDN, which can offload a LOT of content delivery and load on your servers, cloudflare springs to mind. with a distributed CDN you could really cut down you server costs and bandwidth bills. Depends on the nature of your application and the data your pushing to your clients.

    Oh yeah, i was in acronym mode CDN = content delivery network

    • 0

  4. You can use vnstat for monitoring your in & out traffic. This is a console-based network traffic monitor it's maintain the log of hour, daily & monthly also you can use vnstat php-frontend for graphical monitoring.

    Configuration:

    yum install vnstat

    Once installed, you need to create a database with the following command:

    vnstat -u -i eth0 ()

    -u :forces a database update for interface or creates the database if it doesn’t exist
-i eth0 : use to specify interface

    Please note that it will start to collect data via cronjob: 0-55/5 * * * * root /usr/bin/vnstat -u

    for Vnstat PHP-Frontend: http://www.sqweek.com/sqweek/index.php?p=1

    • 0