One of our primary webservers, which runs Apache on Ubuntu, is coming under attack from heaps of referrer SPAM requests lately. There's enough of this that it is really sucking up our internet bandwidth.
How do others deal with this and block it?
Update: OK, thanks to Kevin, I've installed fail2ban and am seeing what I can filter with it. Any suggestions still gratefully accepted though ;-)
There's a really comprehensive .htaccess file here that will deny the majority of those bad bots for you.
Just edit the last line to 'forbid', rather than sending them to the honeypot script...
e.g.: change last line to:
EDIT: Yes "Mozilla/4.0.." spoofing is annoying. If you can't block by user-agent header, then IP addresses have to be the way to go.
For an automated self maintaining solution, have a look at project honeypot!
/ Richy
Is there anything you can use in the referrer request that you could feed to fail2ban?