Home / server / Questions / 349065
Accepted
Jon Cox
Asked: 2012-01-12 15:03:51 +0800 CST

Clustering TCP servers, so can send data to all clients

  • 772

I'm developing a real-time mobile app by setting up a TCP connection between the app and server backend. Each user can send messages to all other users.

(I'm making the TCP server in Python with Twisted, am creating my own 'protocol' for communication between the app/backend and hosting it on Amazon Web Services.)


Currently I'm trying to make the backend scalable (and reliable). As far as I can tell, the system could cope with more users by upgrading to a bigger server (which could become rather limiting), or by adding new servers in a cluster configuration - i.e. having several servers sitting behind a load balancer, probably with 1 database they all access.

I have sketched out the rough architecture of this: TCP server cluster problem

However what if the Red user sends a message to all other connected users? Red's server has a TCP connection with Red, but not with Green.

I can think of a one way to deal with this problem:

  • Each server could have an open TCP (or SSL) connection with each other server. When one server wants to send a message to all users it simply passes this along it's connection to the other servers. A record could be kept in the database of which servers are online (and their IP address), and one of the servers could be a boss - i.e. decides if others are up and running, if not it could remove them from the database (if a server was up and lost it's connection to the boss it could check the database and see if it had been removed, and restart if it had - else it could assume the boss was down.)

Clearly this needs refinement but shows the general principle.

Alternatively I'm not sure if this is possible (- definitely seems like wishful thinking on my part):

  • Perhaps users could just connect to a box or router, and all servers could message all users through it?


If you know how to cluster TCP servers effectively, or a design pattern that provides a solution, or have any comments at all, then I would be very grateful. Thank you :-)

python cluster tcp amazon-web-services twisted

2 Answers

  1. Best Answer

    There are various ways to do this depending on how many servers you ultimately expect to have. Up to about 50 servers, you can pick two or three servers to act as "hubs", each hub connects to every other hub. Each non-hub connects to two hubs.

    Send a message out all connections except the one you received it on. If you receive a duplicate message, ignore it. You'll need to keep track of recently-received messages to detect a duplicate.

    You can track operational servers in the database.

    Alternatively, use a server design that already does this. For example, you can use IRC servers as your broadcast layer.

    • 1

  2. I've been working on a new project that need to make TCP servers distributed. We implement the server in Java. And my solution is using Hazelcast(implemented in Java too) to distribute a Map that contains all Users and a Queue for storing Cluster events.

    With the Map I can easily tell that a user is in which node, so if serverA received a msg from user1 to user2, serverA could look up the Map and turns out user2 is in ServerB, so serverA will wrap the msg in a cluster event with target server as ServerB and put it in the Queue. ServerB will then consume the event and send the msg to user2.

    There could be some impact on the throughput and latency on the whole cluster, but it is tolerable as far as know.

    • 0