For static websites I'm using HAProxy (one master, several nodes with static HTML files on them). What services in addition would you recommend to allow WordPress sites to be hosted in the same way in terms of MySQL and files across the nodes changing.
Thanks for any advice/pointers.
Edit: I'm using Ubuntu currently with a standard LAMP stack, the reason why I need this is because as soon as one of my clients sites goes down because they are getting 1000's of hits is when they call me and I get shouted at.
@polynomial The data changes all the time (the main problem) the slaves need to be access really in a which one has the least load on it. MySQL is currently just a single server, serving the data to WP. We do know a little bit when sites will increase load, but there is not much we can do apart from add more ram and bandwidth.
A wordpress instance with a decent cache (Wordpress SuperCache comes to mind) should help a lot in serving a very heavy wordpress site.
But the first thing you need to do is identify where your bottleneck is. If your database engine is fairly light on, but there's a lot of server-side post-processing of data, then simply scaling out your web servers may help with this, and have them all connect to the same back-end database.
In this scenario, all you need to do is
rsync
your files to/from eachother every few minutes, or if that's not enough, then using some sort of distributed file system (I only know of DFS in Windows, I'm not much of a *nix guy).That's the way Stack Overflow and the Stack Exchange websites work. According to their latest blog post, their infrastructure hasn't changed too much in the last 2 years or so. Just one massive database server for Stack Overflow1 serves a grand total of six Stack Overflow web servers (3 of them dedicated to just SO).
So, if an incredibly complex website like Stack Overflow that serves about a hundred requests per second, 24 hours a day, can get away with a single DB server, I'd suggest that Wordpress with a good caching plugin should be fine for all but the most insane workloads with a single DB server, which makes scaling out much, much simpler (once your infrastructure is set up correctly, just throw more front-end webservers on it).
1OK, so there are two database servers, but the way MSSQL clustering works is that only one database server is ever active at one point in time. So the clustering only provides high availability, not scaled-out performance.