I'm trying to design an architecture to handle massive TCP socket connections and I'm in doubt on the limits of this architecture.
I'll have to handle ~20k concurrent TCP connections and these are long-polling connections, they'll remain connected for long periods of time and will be sending data every minute.
Using threads is out of question, since 20k threads will starve the system resources. I'm planning to use gevent to handle such amount of concurrent connections or even use haproxy and 2 servers (w/ gevent) to handle 10k connections each for instance. Does that makes sense ? Does anyone have some advice or had some experience using gevent with 10K+ connections ? Does anyone have idea of the hardware requirements to handle these connections ? I saw some benchmarks that shows lots of connection timeouts for gevent on 5k concurrent connections, what is not very promising for my problem.
Note: I've already read about the C10k problem and the advices on the "Million-user Comet Application".
Use both at once in combination. Assign something on the order of 1,000 connections to a process. Use a manager to distribute connections and spawn new processes if you reach a point where every current handling process is saturated.
Well, since you can use 2 servers to handle 10k each, why not use 5 servers and handle 4k on each?