Pretty much every page of my website is dynamically generated. However they don't change that frequently (kinda similar to a forum page). So I'd like to cache them using a caching reverse proxy such as Squid, varnish or Nginx.
The problem is that for my logged-in users, each of them will see a personalized header saying "Welcome John Doe. Logout" on the upper right corner of the page (just like serverfault). While users who aren't logged in will see a header that says "Login" instead.
So basically even though every user will see the same page in general, they all slightly different version due to that personalized header.
Is there any way so that I can cache the "main" part of the page and serve it from cache while generate the personalized header dynamically for each individual user?
This must be a very common problem. How is it solved in general?
One thing you can do (besides ESI, which doesn't fix bandwidth issues or reduce the number of requests you must serve) is actually break the page into multiple entities. For example, all of the public stuff is on one cache-able page, then the personalized items are pulled in via an AJAX/iFrame. This is sort of the way that iGoogle works, except in reverse... in iGoogle's case, the base page is customized, but your chosen RSS feed components are public, cache-able, and brought into the main page via AJAX.
With Varnish, you could use ESI, however, ESI processed pages are not able to be compressed without some added complexity. http://varnish-cache.org/wiki/ESIfeatures
I've not used this in production, but, Nginx also provides ESI functionality through a plugin, http://wiki.github.com/taf2/nginx-esi/
Nginx does provide caching through http://wiki.nginx.org/NginxHttpProxyModule#proxy_cache and with the other plugin, would handle both fragment assembly and caching.
http://www.trygve-lie.com/blog/entry/esi_explained_simple explains ESI.
Alternatively, you could do your own fragment caching on the application side.
Yes, cache the main page, but the individual components go in with AJAX calls, and you need to put pipe into vcl_recv for those urls.
if (req.url ~ "/some-ajax-call") { return(pipe); }