What is the best way to share common web application code to many web servers?
For example, I have a directory containing web application written in PHP. Now, I have multiple machines that will serve same content, each runs a webserver, load balanced using TCP/IP balancer.
I haven't tried NFS yet for this scenario. But according to Slow NFS transfer performance of small files, transfer performance will be slow, because of many [small] files involved.
EDIT: I want to store all files centrally in a single location, so update to the code will be instaneously read by the webservers.
I'll outline a couple of options, to show alternatives to the live code via NFS approach.
The "best" strategy depends on your chosen code deployment strategy, possibly one of these:
The "best" strategy also depends on the availability requirements for the application:
Out of the above options, I would personally choose the package deployment approach. However, for web requests, sharing a small number of files over NFS can work well: The latency introduced by NFS is small compared to the Internet. But before you do this, consider the disadvantages:
Because of these possible snags (IO bound fileserver, failure of the webserver), I'd also suggest periodically sync'ing the webservers with the application. Pushing changes to several machines should only take a couple of seconds. If required, you could setup some logic like: if (time() > 23:59:00) {use software in dir B} else {use software in dir A}). This could be useful if all machines must should run the same software version, e.g. if you've just changed a database schema.
A couple of seconds delay during deployment really isn't too bad. A developer working on a live system would certainly notice the delay, but then developers shouldn't edit live systems anyway.
if its content that doesn't change, then any transfer method will do, even NFS. If you tar the files first then unpack them, the transfer rate will be very quick.
Of course, if you need to transfer a lot of small files regularly then you might not have any solution - your web servers will always be out of date with the latest files (or worse, required files).
If you do want to set up a way to deploy files to a central server, and then have them automatically transfer to the others, you could set up a rsync transfer to the servers, then only the files you change will be transferred (if you use the rsync daemon, it'll use rsync's protocol, and will take next to no time at all, seconds in all likelyhood).
A good method others use is to store web files in a version control system and then use that to export the files to the other servers, then you'd just update the files and the vcs would replace changed files with the new version.
Incidentally, NFS isn't too good for small files, but then, nothing else is. The problem is more about network latency than anything else. That still shouldn't put you off using it, unless you have a heap of files to transfer, it won't be too slow to use. You could try using cifs transport instead of nfs, I do this for my Windows server as I got better performance for large files there.
Arie, My webdev company has a large number of libraries that are shared by our websites. The sites run at a handful of Production servers in different datacenters.
Since the servers are at separate datacenters we can't serve the libraries from a single centralized low-latency location. Instead, we use Webistrano (a GUI for the excellent Capistrano deployment tool) to deploy the libraries to all servers, straight from our SVN repo. Capistrano deploys to all servers simultaneously, and only after all servers succesfully deployed it switches a symlink to point to the new deployment directory. Otherwise it does a rollback on all servers.
In other words, the update is performed on all servers at virtually the same second, or not at all. Transactional or atomic or whatever you want to call it.
Our final task at the end of the deploy script is a graceful restart of Apache, in order to flush all PHP caches (real_path, Zend Framework, etc.) that have stored the previous paths.
It works very well, is very safe, and since all files are stored locally on each server latency is very low. It's also very easy; simply clicking Deploy in Webistrano.
You could look at sshfs. Basically you would have one central location which the other servers mount as a local folder.
Eg (to mount the server as a directory ).
Now you can access the other server as if it was a local folder. The added advantage is that data transfer is encrypted.
Google sshfs and fuse.