We have been running nginx -> uWSGI, and now we are evaluating putting Varnish as a caching layer between nginx and uWSGI (similar to http://www.heroku.com/how/architecture).
But, nginx only supports HTTP 1.0 on the back so it will have to create new connections with Varnish for each request.
Many recommend running nginx in front of Varnish, but wouldn't it make much more sense to use something like Cherokee so that you eliminate the HTTP connection overhead since it supports HTTP 1.1 on the back?
We debated this as well when putting in our backend cache layer, we are also using nginx but with squid and a JVM that serves content.
If you aren't using any functionality that is unique to nginx you could switch, we had already built a couple nginx modules.
You should consider what the actual overhead of that connection setup is vs the end-to-end request. For us when testing it was always <2ms even reading an asset from memory cache took more than that to respond (>5ms).