We have an interactive flash application running on AWS EC2 instances. One instance can handle between 100 and 200 users. We expect times with heavy growth and slashdot effects. To be prepared and not over-prepared we want a load balancing solution with automatic scaling.
We tried Elastic Load Balancer but it doesn't seem to work with our application. We experience many and early TCP disconnects.
Right now the best solution seems to be haproxy but it doesn't fit our automatic scaling requirement. We like to have something more programmable/scriptable. We need to add or remove backend servers often.
I'd like to know if there are other - maybe better - solutions.
I guess you could do many things here:
see for more: http://gigaom.com/2010/06/08/how-zynga-survived-farmville/
if you go wo/ them you could use some hardware solution: citrix/cisco/juniper all have some solution for LBs
software LBs: check nginx it is a really nice project, i am not sure if they added the LB features lately but with a bit scripting you could use it and it is damn fast and efficient
Look at Varnish; its configuration language is sufficiently flexible that you ought to be able to coerce it into what you want to do.