Nginx will only use error pages with certain filenames

Using Nginx I am able to get the default error pages to work for 404 and others, but only if I name the file exactly after the error code, i.e. 404 -> 404.html... 40x.html doesn't seem to work.

I want to be able to diplay the same error page for a lot of different errors, and I've seen something like this done:

error_page 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 421 422 423 424 426 428 429 431 451 500 501 502 503 504 505 506 507 508 510 511 /error.html;

https://blog.adriaan.io/one-nginx-error-page-to-rule-them-all.html

but I can't seem to get it to work. Any ideas? Renaming the file (and in the config) breaks it.