Let's say I have a load balancer in front of 3 app servers.
Let's say I also have these services available at certain IPs:
- Postgres server
- Redis server
- ElasticSearch server
- Memcached server 1
- Memcached server 2
- Memcached server 3
So that's 6 nodes at 6 different IP addresses. Naturally, every one of my 3 app servers needs to talk to these 6 servers above.
Then, to make it a bit funkier, I also have 3 worker servers. And each worker also talks to the above 6 servers, but thankfully workers and apps never need to talk to each other.
Now's the kicker. Everything is on Digital Ocean VPS. What that means is: you have no private network, no private IPs. You only have separate, random IP address on each machine. You can't mask them or anything.
So in order to build a secure environment I would have to configure some iptables. For example:
- Open app servers be accessed by load balancer server
- Open redis, ES, PG, and each memcached servers to be accessed by each app's IP and each worker's IP
This means that every time I add an app or worker I have to also reconfigure iptables in those above 6 servers to welcome the new app or worker.
Is there a way to simplify this type of setup?
I was thinking — what if there was a gateway machine between apps/workers and the above 6 machines. This way all the interaction would always happen via the gateway server, and when I add a new app or worker I wouldn't need to teach the 6 servers to let it in.
If I went this route, then I'd hope a small 512mb server could handle that perhaps, and there wouldn't be almost any overhead. Or would there?
Please help with best way to handle this situation. I would appreciate an answer as concrete as possible. I don't think this is too specific, because this general architecture is very common, and Digital Ocean is becoming increasingly popular. A concrete solution here would be much appreciated by many.
In case someone else is looking for solution here, no there is no solution. You have to manage many-to-many relationship between nodes, and reconfigure their respective iptables and hosts. A gateway machine approach might work, but I didn't go with it.