How To limit Nginx Auth_Basic re-tries?

As far as I know, Auth Basic module doesn't support this feature, but you can do this by using Fail2ban.

Testing with a non-existent user, you will see something like belows in the error log:

2012/08/25 10:07:01 [error] 5866#0: *1 no user/password was provided for basic authentication, client: 127.0.0.1, server: localhost, request: "GET /pma HTTP/1.1", host: "localhost:81" 2012/08/25 10:07:04 [error] 5866#0: *1 user "ajfkla" was not found in "/etc/nginx/htpasswd", client: 127.0.0.1, server: localhost, request: "GET /pma HTTP/1.1", host: "localhost:81"

Then create necessary filter:

/etc/fail2ban/filter.d/nginx-auth.conf

[Definition]
failregex = no user/password was provided for basic authentication.*client: <HOST>
              user .* was not found in.*client: <HOST>
              user .* password mismatch.*client: <HOST>
ignoreregex = </host></host></host> 

/etc/fail2ban/jail.conf

[nginx-auth]
enabled = true
filter = nginx-auth
action = iptables[name=NoAuthFailures, port=80, protocol=tcp]
logpath = /var/log/nginx*/*error*.log
bantime = 3600 # 1 hour
maxretry = 3

Testing Fail2Ban rules:

fail2ban-regex /var/log/nginx/localhost.error_log /etc/fail2ban/filter.d/nginx-auth.conf

Failregex
|- Regular expressions:
|  [1] no user/password was provided for basic authentication.*client: <HOST>
|  [2] user .* was not found in.*client: <HOST>
|  [3] user .* password mismatch.*client: <HOST>
|
`- Number of matches:
   [1] 1 match(es)
   [2] 2 match(es)
   [3] 0 match(es)

Ignoreregex
|- Regular expressions:
|
`- Number of matches:

Summary
=======

Addresses found:
[1]
    127.0.0.1 (Sat Aug 25 10:07:01 2012)
[2]
    127.0.0.1 (Sat Aug 25 10:07:04 2012)
    127.0.0.1 (Sat Aug 25 10:07:07 2012)
[3]

PS: Since Fail2ban fetches log files to ban, make sure logpath matches with your configuration.

I'm amazed no on else has given this solution/workaround.

Nginx basic-auth and htpasswd support bcrypt password encryption with an optional cost variable. Bcrypt is designed to be slow, thus providing a hard limit on how fast you can attempt different passwords.

When creating your basic auth username/password use

htpasswd -B -C 12 path/to/users.db <username>

With a cost of 12 your server will likely not be able to try passwords more than a few times a second, increase that to say 14 and you'll probably be looking at around 1s per password attempt.

With that configured any reasonable password will be immune to brute force attack even if the attacker tried passwords continuously for years.

E.g. at 10 password attempts per second brute force attack on an 8 character alphanumeric password would take 692,351 years: 62**8 / (10*3600*24*365).

This is much easier to configure and more fool-proof than setting up "intelligent" request limitting.

I don't believe nginx has any internal facility to do this. The documentation page doesn't suggest it's possible.

You can use Fail2Ban to block IP addresses that have repeated failed login attempts.

The Fail2Ban wiki has some nginx-specific patterns.

Fail2Ban should be available as a package on most of the big distros.

Nginx-HTTP-Auth-Digest module can replace the basic auth module with many additional feature suck as retry and timeout. Additional documentation is available here

The only downside is that this probably require rebuilding nginx

This can be acomplished with a combination of NGINX basic auth and rate limiting.

http {
    map $http_cookie $rate_limit_key {
        default $binary_remote_addr;
        "~__Secure-rl-bypass=SomeRandomBytes" "";
    }
    limit_req_status 429;
    limit_req_zone $rate_limit_key zone=auth:10m rate=1r/m;

    server {
            auth_basic "Private Content";
            auth_basic_user_file your-auth-file.txt;

            limit_req zone=auth burst=20 delay=10;

            add_header Set-Cookie "__Secure-rl-bypass=SomeRandomBytes;Max-Age=${toString (3600*24*180)};Domain=$host;Path=/;Secure;HttpOnly";
    }
}

This snippet was extracted from my working config, but the snippet itself wasn't tested.

The basic concept is that you create a cookie that allows bypassing the rate limit then set the cookie once someone successfully authenticates. This way someone is rate-limited until they login at which point they can perform as many requests as they want.

The major downside to this approach is that the cookie is static, once someone is logged in on any account they can use the token to brute force other accounts. This is not a major issue if your users are trusted not to brute force the password of other users. However I don't think you can do better than this in NGINX's config. Ideally the cookie would be hash("$remote_user-SomeRandomBytes") so that the token is bound to the user that successfully logged in.

Just to chime in my own solution to this problem: It is based on an other (popular) answer, similar to your problem. Here is the configuration:

limit_req_zone $binary_remote_addr zone=login:10m rate=10r/m;

server {
    listen 80;
    server_name XXX;

    location / {
        auth_basic "Authentication required.";
        auth_basic_user_file /path/to/htpasswd; # You should change this
        error_page 401 /AuthFailureLimit/index.html;

        root /usr/share/nginx/html/normal; # Demo
    }

    location /AuthFailureLimit {
        internal; # To prevent visible redirects
        auth_basic off; # Just in case the "auth_basic" was used in the "server" section

        limit_req zone=login burst=20 nodelay;
        limit_req_log_level warn; # Show blocks on the console
        limit_req_status 429; # Make more sense than "503 Service Unavailable"
        
        # ONLY use root or alias in this location, other statements (like return) will be executed before the rate limit triggers!
        alias /usr/share/nginx/html/ratelimit; # Demo
    }
}

It allows you to directly utilize the rate limiting feature of NGINX, without requiring the client to store e.g. cookies. Please note the comments in the configuration itself, as (due to the different NGINX phases) some statements may mess up the rate limits.