I have a rapidly growing (10Ms calls/day) web service currently on one EC2 app instance running ubuntu10.04+ apache + APC + memcache (+ several master/slaves mysql farms on other instances).
I consider to upgrade the app instance in one of the following ways:
1) Increase to larger instance type
2) Split to several app servers
I can assess pretty well my memory usage and cpu usage based on the current server graphs.
My fear is that if I go for the larger EC2 instance type the network i/o may become a bottleneck as traffic grows.
My questions: How can I estimate network i/o usage and constraints? How can I tell how much spare traffic I can handle until it becomes overloaded. (in terms of the network capacity not the server ram/cpu)
I used to have CentOS/plesk server on Mediatemple and there my bottleneck was kmemsize which is a limited amount of nonswappable memory which became full when the network traffic increased. Is there something similar to this on EC2 Instances with Ubuntu that can be monitored?
I don't have a full answer for you, but I can build on what cyberx86 said. The smaller instances have less CPU and network capacity. Remember that if you're using EBS that your disk IO is dependent on your network IO. Amazon provides tools for monitoring your network usage. Tools like iotop should be able to help you get an idea of disk IO, which can also show your wait %. Unless I'm mistaken, wait % for EBS volumes should be network constrained.