Imagine we have a site made of nginx as a frontend, and apache (with php or whatever) as a backend. The idea is that nginx will serve static while proxy requests to *.php files to backend. Easy to implement and no-brain-needed config, isn't it?
Now imagine we have bad-behaviour backend. It uses some unstable modules, it used to produce pages for seconds and we're not sure if the page be rendered at all. Weird, but still possible, right? Legacy code and so on. By the way, the backend may be on different machine (yes, legacy: old OS, old software, "don't even touch it!" etc.)
Now we'd like to "protect" ourself in a case the backend won't answer. It would be great to serve page as we served it last successful time when we got no page or get 5xx error. To say in other words, if backend returned 200 code and the page content, then we serve it. If we got timeout or we got 5xx code from backend, we serve the page for the same URI but cached one. This will give us some time to fix backend while users will see at least something.
Nginx is a nice piece of software. I found proxy_cache_use_stale directive and also some others but I'm afraid what will be the version that will be served if backend fails.
I can also imagine some http balancer (nginx as an example) that will check the backend status and stop use it whatever happens. The second backend then maybe server full of static pages that mimics the original site (maybe even without some element, like 'login' link). If so, how should I wisely update the 'mimics' server over time?
What approach may be better? What would you choose?
CloudFlare have 100+ data centers all over the world, they're very well placed and very well peered, and you can't beat the price of their free tier. Yes CloudFlare does show a warning if the main site is offline, but maybe their paid business plans can suppress that warning - you'd have to ask them. If you don't want the warning your only option is Nginx or similar caching.
They key parts of the nginx configuration would be these, in your location block. There's a lot more about them in this excellent article. Basically it says only cache 200 (success) responses and only for 1s (increase that if you can), use the stale page while you're updating, and don't queue requests for pages already requested.
If you have anonymous users page caching will reduce the load on your server, even if you only cache pages for 1 second, though longer is better. This reduced load may make the application more reliable.
Of course you should fix the badly behaved application.