My web server is getting a lot of GET request on thinker/js/think.js and thinker/showSimilarInfo.do.
These requests obviously constitute a server attacks.
What is point of these attacks?
Also, is there a way to automatically ban IP which is trying to this kind of attack.
Is there are a way to automatically block IPs with apache2. For example, if we get a lot requests (100s) for non-existing pages (or known patterns, like CHANGE_LIST.txt, etc.) to block that IP and add it to deny list.
On my Apache-hosted Ruby on Rails website, an IP address (always identical) from Russia was hammering the website every twenty minutes, based on the path requests I think it's trying to seek out comment forms to submit spam comments onto.
Anyway, I did the following in my .htaccess file to completely boot them off while allowing everyone else...
Order Allow,Deny
Deny from XX.XX.XX.XXX
Allow from all
So it worked, their IP no longer shows up in my Rails logs and they can't access the site. However, the error log for the website in the /var/log/apache2 folder continues to rack up the same error by the banned IP...
[Thu Jun 30 09:11:37 2011] [error] [client XX.XX.XX.XXX] client denied by server configuration: /srv/[...]
It's obvious what the error is for, but I don't think it's necessary to keep recording that this IP was blocked, I want to keep my errors file clean for other legit errors. How can I stop the logs from recording this?
The problem:
I manage a website which has lot's of dynamically generated pages. Every day bots from Google, Yahoo and other search engines download like 100K+ of pages. And sometimes i have problems with "hackers" trying to massively download the whole website.
I would like to block ip addresses of "hackers" while keeping search engine bots crawling the pages. What is the best way to do this ?
Note:
Right now i am solving the problem as follows. I save ip of each page request to a file every X seconds. And i have a crontab script which counts repetitive ip's every 30 minutes. For ip's which are repeated too many times, the script checks a hostname- if it doesn't belong to Google/Yahoo/Bing/etc then we have a candidate for banning.
But i don't really like my solution and think that auto banning could be done better or using some out of the box solution.
OK, question: I have a IIS webserver and Windows firewall. To allow incoming connections to IIS, I enabled port 80 and 8080.
But now, how can I block a specific IP address/range on port 80 (=ban) with windows firewall ?