Make nginx show 404 page for all urls that don't match a pattern

I configured nginx as a reverse proxy for some app urls, like /auth, /api, /users and others. I want nginx to reject all the other requests and show 404 or do something else.

I have this in my config now:

server {
    listen 80;

    error_page  404  /404.html;

    location /404.html {
        root static;
    }

    location /auth$ {
        proxy_pass http://localhost:8080/auth;
        proxy_redirect off;
    }

    location /api$ {
        proxy_pass http://localhost:8080/api;
        proxy_redirect off;
    }

    location /users$ {
        proxy_pass http://localhost:8080/users;
        proxy_redirect     off;
    }
}

When I visit any of matched urls, all works good, but when I enter something like /abrakadabra, it shows a loading spinner, trying to load something, then shows empty page even though I have the file for it, and in console I see it's 404 error.

I see these requests in access.log.

What am I doing wrong and how to fix it?