I have a deploy script where I want all of my webservers to simultaneously perform a git pull from the same place. When only two servers do the pull simultaneously, there are no problems. Once I have three or four servers, then it often happens that one or more servers 'times out' and doesn't finish the pull. I am using git over ssh.
Is there any server software I can use that will manage the incoming requests and and deal with each one individually or something?
Thanks
You're likely just hitting some limit on the server hosting the master git repo. You don't provide any data on this to help with diagnosis, but if you're on a slow/cheap server, I'd start looking at possible IO contention.
Do all the web servers really need to pull at the exact same time? If not, just add a bit of "fuzz" to the script by having each sleep for a random number of seconds (say between 0 and 15) before doing the pull.
If that's not an option, try doing a manual git pull from your workstation (to a repo that's at in the same state as the web servers) before kicking off the deployment script. This will prime the server's disk cache and hopefully it will be able to serve most of the pulls out of cache instead of hitting the actual disk.