Why is Nginx so fast?

You may see this presentation for an overview of nginx internals. The main difference is asynchronous handling of requests instead of using threads as Apache does. You may have a look at this documentation as well.

How does a site like rambler serve dynamic content so fast? ... Is this purely Nginx’s capability? Where should I be looking into to learn about such capabilities?

This has little to nothing to do with the web server used -- both nginx, IIS and Apache are 'fast enough' an generally do their work within milliseconds. nginx is much faster than Apache, but this merely means the site owner will need fewer servers for the web serving part -- nginx does not transfer data faster to you.

The less important part is server-side speed, i.e. the time it takes to create the HTML. The more important part is the 'frontend' performance, by which I mean the HTML, CSS, Javascript and Images, the number of these, the size of these, and the proper delivery (HTTP compression, caching) of these.

Of course the server-side speed is still important, I'm not saying it should be ignored or that it doesn't matter. But typically it is the smallest part perceived of end user speed -- the serverside work is often done in less than 500 milliseconds, but the page isn't ready before 3,000 - 5,000 milliseconds have passed. The bulk of this time goes to downloading the frontend resources (CSS, Javascript, Images).

Steve Souders did the original work while at Yahoo, he is now working at Google. His first book "High performance websites" is the best starting point for learning more about making fast websites. The same material that's in his book can be found in this video talk, and these design rules. However, I find that the book is quick to read, and much easier to understand.

You can run the sites through WebPageTest.org's tester -- that will give you a good feeling for the frontend part of these sites, and why they are faster or slower.

I believe that serverfault.com if served from Nginx will be much faster the IIS 7 (assuming db access time to be same in both the case). Is this a fair assumption?

Nope, that is a misunderstanding. :-)

Nginx is more often used for load balancing other applications/servers and serving static content than it is used as the complete server.

For instance you may write an app using one of the many python frameworks, and have nginx be the front-end to many instances of that (perhaps spread over several machines). In this case nginx servers two purposes: it handles requests for static content like images and stylesheets directly (and due to its design it does this very quickly), and it passes dynamic requests to the application spreading the load between all the instances it knows about. This is a very popular configuration in the Ruby on Rails community too.

There are two other posible reasons why Rambler may appear faster to you than the local Yahoo service. Firstly the local Yahoo PoP might just not have enough resource available to serve the number of requests it gets any quicker so maybe simply adding more hardware (assuming the software scales well this way) would speed it up (but, presumably, the difference is not worth the cost of maintaining the extra kit or Yahoo would have done this). The other big difference may be in the back-end rather then the web server - the two services will no doubt have very different database arrangements and even if not they are not likely to be running exactly the same variety of queries (and the amount of hardware dedicated to the database archicecture will have a significant effect too).

Analysing why one service is faster than another (generally or in specific circumstances) is usually not going to result in a single simple answer - there are many ways to design an app that is intended to scale out to many thousands of users, each with its own benefits, problems & compromises and even if you factor all those differences out each site will have a different user-base dynamic, plus there are network issues beyond the control of the designers too.

nginx possibly but more scalable architecture with reasonable load balancing in front of static content servers / dynamic content generators. if you really want to get great end-user experience you should probably move content closer to the 'eyeballs' - use some CDN.

if you're interested in the subject - check out this and that and.. well - google ;-]

The best sites use application accelerators such as Zeus's ZXTMs - they can cache dynamic responses in many cases which is obviously of great benefit.

Here is a good explanation: http://www.aosabook.org/en/nginx.html

I have a hard time seeing serverfault much faster (SO might have load problems due to traffic perhaps?) as it's already an instant page load here in EU over my route. It's way quicker and more responsive than most local news sites and so on.

Most of the obvious problems with load times and latency comes from between the server and the end user imo and not the actual server performance (unless someone sized or designed something wrong). Different sites can be routed different ways and there's a big possibility that a country-local site for me has a bigger latency than something across the planet - it all depends on so many variables that you can't say that it's solvable by just a service upgrade/switch unless you know that's where the issue for a specific use(r) is...

Obviously caching of various kinds on the server makes a big difference but all these sites already do that as far as I know.