My company has a web service api which is starting to get a lot of use. Recently we had some issues with running out of memory. We optimized some inefficient code and solved the problem.
We know we are going to expand even further, we want to have a good way to deal with the intense traffic.
One idea that has come up is to have different url's for some of our heavier customers. This just jumps out as the wrong thing to do to me. The url's in some cases would point to isolated servers, but some would also just point to more virtual directories.
Is this a good solution to the problem in either case? I'm forseeing horrible maintainability issues and causing more problems then it solves. Please give me some pros and cons for both sides.
This is already on a load balanced server farm.
If it's already on a load balanced farm, and you're getting too much load, and you've already optimised as much as you can, the natural next step would seem to be expanding your farm to meet demand.
If you are hitting the maximum capacity of the load balancer, and you have idle servers, you can try to balance more evenly using some sort of feedback, with balancers like mod_cluster. If you are still hitting the limits, you can try Round robin DNS as an alternative to handing out multiple URLs. This way you can offload the load balancing to the client. You can add feedback to this solution with lbnamed. A bigger load balancer is another approach, which of course requires more $.
Can your API take advantage of caching?
If specific parts of the API are getting called often, and returning the same results something like memcached might help you significantly.
I don't see an advantage to having specific urls for different customers. It seems to me that either you need more servers and/or your load balancing isn't working correctly.
A layer 7 load balancer would allow you to segregate those clients that are considered 'high cost' to a cluster/machine that is set up to handle their specific queries. I'm assuming that it is a 'busy' client that is doing the lions share of requests, and splitting them to their own server is why you considered giving them a separate url. With a layer 7 load balancer, linuxvirtualserver.org, you could filter particular URLs and have a rather easy to maintain system.
While you ultimately want to fix the problem the right way, using something like this might buy you enough time.
I would be looking at those heavier-usage customers right now. Can you charge them more? Is their usage more than it should be? Can you help them optimise their systems so that they don't need to be making such heavy usage? Can you add rate-limiting to your APIs? Is their usage enough to be able to provide them with their own server and charge them accordingly?
Next step is to look at your architecture. Do you have caching proxies in front of your web servers? Do you have separate database servers? Are all your servers optimised as well as your code? Are you caching absolutely everything you possibly can? Do you have good hardware?
Once you are sure your architecture is as optimised as possible you really just need to be adding extra boxes to the load-balancing setup whenever you start hitting limits (whether that's extra caches, extra web servers or extra backend servers will depend on what limits you're hitting).
Have you tried running http://developer.yahoo.com/yslow/ and http://code.google.com/speed/page-speed/ that are plugins to Firebug, trying to analyze the number of requests generated when a page is loaded?