I'm trying to get an overview of how i would go about creating a load balanced web server setup. Setting up the actual load balancer and adding two or more web servers seems fair enough. However, I can't decide what the best setup for files would be.
The web servers will be running a CMS system (in this case Drupal). There might be files uploaded by users which relates to content, and this should be accessible to all web servers. Now, I was thinking of simply having a designated file server and mount a file directory on each web server so that the various web servers could access these "shared" files.
However, would this solution be a bottleneck if we're talking about 4-5 servers? I'd think the mounting would be done over SSH.
All help highly appreciated!
Update (January 15, 2013)
I decided to go with a combination of Csync2 and Lsyncd. I then set up two servers which acts as mirrors. Lsyncd listens for changes in certain directories (and their subdirectories) and invokes Csync2 which takes care of synchronizing files to the other server. If you're interested in knowing more, have a look at this tutorial.
John and Cailin's five-step tutorial on scaling Drupal will be very helpful to you.
The first two steps are as follows:
Step 1: Separate your web server from your database server.
Step 2: Set up redundant web servers and load balancing.
Bottom line: you are not going to want to scale horizontally (adding multiple servers with identical functions) without first scaling vertically (separating functions that currently reside on the same server into different tiers).
Before I moved to the CDN I had a web farm of about 20 servers. I started with NFS until I hit a bug in the linux kernel about 3 years back that caused an issue with not mounting the volume and it wasn't failing over correctly.
I then created an rsync script that would sync the dir's every 5 minutes to each image server. This wasn't ideal but it worked and I didn't have any issues with it.
The next step I switched to lsyncd which uses inotify to monitor a directory and when a file is changed it spawns a process to sync that file. By default that's rsync. That solution was solid and if not for the economy of a CDN I would have continued with that.
Lsyncd: http://code.google.com/p/lsyncd/
Hope that helps.
I have used glusterfs (http://www.gluster.org/) to share files between load balanced web servers. I tried to also share the MySQL database files between the servers as well with glusterfs, but with no success - I recommend to separate the web from the database server as the first poster suggests.
Other solutions to share files between servers are lustre (http://wiki.lustre.org/index.php/Main_Page) and, depending on the load balancer setup, you could also use lsyncd (http://code.google.com/p/lsyncd/) to synchronize files between the servers.
You can set a highly available NFS share and mount it on every http server to serve your files.
Please find a tutorial for a highly available MySQL cluster: http://www.howtoforge.com/loadbalanced_mysql_cluster_debian