Equivalent to denyhosts, but for HTTP requests?

http://www.modsecurity.org/ may do what you want.

fail2ban scans log files like /var/log/apache/error_log and bans IP that makes these automated scans based on regular expressions (called filters). By default it updates the firewall (iptables) to block the offending IP. It is very easy to write new actions and implementing one for updating a .htaccess should be quite simple, there are several examples available in the fail2ban distribution.

I'll second fail2ban. It works live, it can ban temporarily, and it can add IP addresses to your firewall so Apache doesn't have to waste time on it.

Even more efficient when coupled with the ipset netfilter module for iptables (which is faster for handling large amounts of addresses), and it can ban them instantly so they only get to make one or two requests before getting blocked.

If you really, really hate these people and are running Linux, you could also try to implement tarpitting for iptables (a quick search hasn't found any 2.6-compatible patches). This will accept the connection and then immediately set the window size to 0 (preventing data from being transferred), but also prevent the remote end from closing the connection at all, meaning whatever app is connecting will have to wait somewhere between three and twenty (!!) minutes before the connection times out on their end.

This is great for stopping portscanners as well, because it makes them take orders of magnitude longer than they normally would.

If it's Apache 2.0, you can try mod_access: http://httpd.apache.org/docs/2.0/mod/mod_access.html

In Apache 2.2, it's mod_authz_host: http://httpd.apache.org/docs/2.2/mod/mod_authz_host.html

It sure looks like you're going to have to signal Apache to reload its configuration to make changes in the module's configuration take effect, though.

Edit: Modsecurity looks like what you want, as opposed to building static deny configurations by grepping your log files and using one of the above. I'm gonna upvote that answer now.

Another possibility is using mod_rewite to match the URL and sending the client a 403 (or whatever code you want). Example:

RewriteEngine on
RewriteRule ^/.*passwd.*/ - [L,NC,G]

Or for fun:

RewriteEngine on
RewriteRule ^/(.*passwd.*)/ http://127.0.0.1/$1 [L,NC,R=301]

I would certainly look at OSSEC. It detects these patterns and can block the ip addresses. It also looks for scans (multiple 404s for example) and block the offender IP.

And yes, it does that by default.

Link: http://www.ossec.net

An example would be:

<Location />
   Order Deny,Allow
   Deny from 123.123.123.123 231.213.123.0/24
   Allow from all
</location>

I'd recommend having this in a separate file that you include from your main config and just rewrite this small fragment. You can do a reload to pick up the change, which doesn't stop your server like a restart would.