I'm working with an organisation that has deployed Varnish as a caching reverse proxy for all of their web traffic. Their traffic make up is a lot of customer-generated dynamic websites, with the usual collection of static assets hanging off the side.
While I'm trying to like varnish (I think it has a pretty good architecture, in principle), I'm having some troubles managing it and troubleshooting problems as they arise, so I'm wondering if it's really the right choice. I've used squid in the past as a reverse proxy, but not in the same sorts of role, so I don't have a clear basis for comparison.
My question is targeted at people who have deployed varnish in production or seriously evaluated it against the alternatives: did you stick with varnish, or did you end up using another reverse proxy? What were your key points for staying with it or switching, and if you did use something else, what did you end up using?
Well, I'm running Varnish on my webservers, primarily for performance reasons, although its load balancing featues are handy as well.
My use case is caching in front of Django-based websites, and it does wonders for page loading performance. I'm able to serve most pages directly from cache and handle a flood of visitors with little trouble.
The reason I chose Varnish was mainly performance/scalability. The main points:
In my experience, Varnish performs a bit better than Squid in most cases, and a lot better on traffic spikes. On the other hand, configuring Varnish correctly is going to take some mailing-list-trawling, since there aren't as many ready-to-go-for-your-specific-use-case-documentation flowing around the net as there are for Squid – mainly due to Varnish being a fairly young project in comparison.
I spent a long time trying to get Varnish 1.x stable on bog standard linux/dell hardware, it would always hang in some odd way and its watchdog would restart it. Which was fine, except for the hard won cache that was not being persisted anywhere else...
Having said that, are you really using the right tool for the job? If you want a reverse proxy that will cache the results of the request (assuming you are providing good quality cache headers) then varnish is a good option. Hopefully it's got more stable in version 2.0
However, if you're running a *onRails site and you want load balancing over multiple backend servers then HAProxy or Nginx may be the way to go. If you don't need any complicated url logic (redirects, regex matches to rewrite older urls, etc) then HAProxy will fit the bill. If you do need something more capable, then give nginx a go.