Home / server / Questions / 66003
Accepted
Daniel Magliola
Asked: 2009-09-17 09:51:05 +0800 CST

Is it normal to get bandwidth as the first scalability bottleneck with a 100 Mbps connection?

  • 772

I'm stress testing a site that we're making, and we're finding a very surprising result compared to my expectations:
Our site starts to load very slow with a few hundred simultaneous people, even though CPU and memory are fine. Looking at Task Manager, Networking tab, I see that my 100 Mbps network card is maxed out at 98%.

For some reason, this sounds extremely ridiculous to me...
Every time I read something on scalability it's CPU, Memory, Caching, etc, etc, and here I'm getting the bottleneck on the network card itself.

We serve all our content gzipped, and our home page is kind of heavy, but not THAT much. I would've never expected the network card to be the bottleneck.

Is this normal?
Is everyone having public facing websites using a 1Gbps network card?
I thought 100 Mbps would be the standard.

Am I looking at something wrong? Am I interpreting the graph in the Networking tab incorrectly?

NOTE: I can think of a number of ways to fix this, starting with getting a 1 Gbps card, and moving static files to their own server(s). My question is mostly around whether everyone is simply using 1 Gbps connections, which would surprise me enormously.

networking

4 Answers

  1. Bandwidth as the first bottleneck is not something I'm too surprised about. CPU, RAM, HD and all the other components have all come on in leaps and bounds over the years, whereas 100 Mbps has been there for over a decade now. So you're in a situation where you've got a good box that's more than capable of handling a typical load, but it's connected using decade+ old technology.

    Even so, are you quite certain that your 100 simultaneous user simulation is an accurate reflection of what real world traffic would be like? With 100 absolutely simultaneous hits, it only requires to serve 1 megabit each, or 128K each, for you to hit peak traffic. That's a very low ceiling, and my feeling is that - unless you're certain you're going to be getting that kind of usage - you might need to revise your load testing.

    • 3
  2. Best Answer

    It sounds exactly like you are saturating your available bandwidth. You either need to cut down on your bandwidth or switch to a 1Gbps card, which is what I'd normally expect to find in a publicly facing web server (certainly has been the case with every server class machine I've touched in the last 10 years - where did you find a server with a cheapo 100mbps card anyway? Is it really a repurposed desktop?).

    Some things to check or consider:

    • You don't mention caching. If your site is setup in someway that does not return good caching headers for static files (such as images) you will take a large hit. Use FireFox and the YSlow addon (from Yahoo) to show you the pie chart comparing cached and uncached page size.
    • What is your testing methodology? Are your 100 "users" just hitting the site as fast possible? What about caching - if your users are just bots that keep grabbing some page, they may be ignoring your caching hints (see last point).
    • You are using gzip compression, but how much of your content is text (gzip helps) and how much is images and other binary files (gzip usually does nothing)?
    • Are you using your network bandwidth for any other functions - such as a seperate database server?
    • You are not really specific about how big your pages are (use YSlow to find out). Are you maybe using large images in place of thumbnails (I've seen my fair share of sites that have several megabytes worth of images on one page because the designer/designer's tools just used the width and height html attribute to downsize the image files into thumbnails).
    • 2

  3. For years the simpler, less capable web servers have been touting their speed, and for years Apache aficionados have been noting that Apache is fast enough to easily saturate the network interface. Sounds like you have an efficient site. Are you really pumping 100 megabits, or is the network stack just taking up a lot of CPU?

    • 1

  4. I'd recommend to check the network side of your environment:

    • there is no broadcast storm or stupid chatty protocols eating your bandwidth?
    • check if the switchport is running at 100/full (don't know if you can check that on Windoze). Half duplex is really bad.
    • see if the error counters on the switchport are >0. if so, replace cables and/or NICs and try another port.

    If there are no problems, upgrade to gigabit (which should be the standard for servers anyways).

    • 1