Is it normal to get hundreds of break-in attempts per day?
772
I just checked my server's /var/log/auth.log and found that I'm getting over 500 failed password/break-in attempt notifications per day! My site is small, and its URL is obscure. Is this normal? Should I be taking any measures?
In today's internet this is quite normal sadly. There are hordes of botnets trying to login to each server they find in whole IP networks. Typically, they use simple dictionary attacks on well-known accounts (like root or certain applications accounts).
The attack targets are not found via Google or DNS entries, but the attackers just try every IP address in a certain subnet (e.g. of known root-server hosting companies). So it doesn't matter that your URL (hence the DNS entry) is rather obscure.
use strong passwords everywhere (also in your web applications)
for SSH, use public-key authentication if possible and disable password-auth completely (howto)
Additionally, you can install fail2ban which will scan the authlog and if it finds a certain amount of failed login attempts from an IP, it will proceed to add that IP to /etc/hosts.deny or iptables/netfilter in order to lock out the attacker for a few minutes.
In addition to the SSH attacks, it is also becoming common to scan your webserver for vulnerable web-applications (some blogging apps, CMSs, phpmyadmin, etc.). So make sure to keep those up-to-date and securely configured too!
I for one use a "tarpit" in addition to only allowing public key authentication and disallowing root logins.
In netfilter there is a recent module, which you can use with (INPUT chain):
iptables -A INPUT -i if0 -p tcp --dport 22 -m state --state NEW -m recent --set --name tarpit --rsource
iptables -A INPUT -i if0 -p tcp --dport 22 -m state --state NEW -m recent --update --seconds 180 --hitcount 6 --name tarpit --rsource -j DROP
iptables -A INPUT -i if0 -p tcp --dport 22 -j ACCEPT
What that does is, that every attempt to connect to port 22 is listed by the recent module with IP and some other stuff under the name "tarpit" (if you're curious, look at /proc/net/xt_recent/tarpit). Obviously you can use other names.
This rate-limits the attempts to 5 in 300 seconds. Please note that users with an existing connection are not bothered by that limit, because they already have an established connection and are allowed to create more (even above the rate limit).
Adjust the rules to your liking but make sure that they be added in that order (i.e. when adding then use them in this order, when inserting then in the reverse order).
This cuts down the noise immensely. It also provides actual security (against brute-forcing) unlike the perceived security of changing the port. However, I'd still recommend changing the port if it's feasible in your environment. It will cut down the noise level a lot as well ...
You can still combine this with fail2ban, although I've been running just fine without it and only the above rules.
EDIT:
It is possible to lock your self out doing this, so you can add something like the following that lets you clear you ban by knocking on a particular port:
iptables -A INPUT -i if0 -p tcp --dport <knockport> -m state --state NEW -m recent --name tarpit --remove
You could implement fail2ban, or similar methods like locking SSH to your IP. Sadly bots attempt to bruteforce access all the time so it is quite normal, you need to make sure you have a good password.
Please use only public key authentication for administrative purposes if possible. Generate a private key on you workstation:
$ ssh-keygen -t dsa
Copypaste the contents of ~/.ssh/id_dsa.pub to you servers ~/.ssh/authorized_keys (and /root/.ssh/authorized_keys, should you require direct root login).
Configure your servers /etc/ssh/sshd_config to only accept public key authentication:
PubkeyAuthentication yes
PasswordAuthentication no
PermitRootLogin without-password
If you have too many servers, you can use Puppet to run public keys and configurations to them.
Look into Denyhosts and fail2ban to block repeated SSH login attempts and see Snort if you require full IDS/IPS.
The attempts are mechanized, so the numbers seem OK (yes they are high compared to some sites and the low compared to others). You should take the steps you normally have to: You consider your sites as attack targets every day, even when you do not detect an attack; not detecting an attack, does not mean it does not exist.
this is common, but that doesn't mean you shouldn't fight the good fight. Here are some steps on how you can make your server more secure.
Avoid DNS associated IP addresses
You can greatly reduce this number in shared or colocation environments by disabling SSH access on any IP addresses associated with domain names. Unlisted non-domain IP addresses will receive less of this type of traffic, so buy an unlisted IP and only use this IP for SSH access.
Use a VPN for all SSH access
If you are in an environment in which you can implement IPsec/VPN to a private network within your server environment, this is ideal. Disable all SSH Internet access, make sure you have an integrated lights out solution. Setup your VPN, and only allow SSH access from your VPN.
Implement IP address rules for SSH access
If the VLAN isn't an option configure your router, or firewall rules to only allow SSH connections from a known IP address range.
If you follow these steps you will sleep much easier in the night knowing someone would have to compromise your hosting companies network to gain access to the server through SSH.
At a previous employer, one of the computer security researchers called the constant stream of break-in attempts the "internet equivalent of cosmic noise". He described it as a normal, continuous flow of malicious traffic that was searching for systems on the internet and automatically exploit scripts to attempt to hijack the system. Bot-nets and other malicious systems would perpetually scan and rescan the internet for vulnerable systems much like SETI.
In addition to using an automated lockout mechanism like fail2ban you have one more option: actually contact the abuse address ISP of the attacker. It may seem completely futile but in the case of the script-kiddie their ISP is more than willing to take action on them.
To find the abuse address, start with arin.net and lookup the IP address using whois. You may be redirected to another regional registry but eventually you can find the responsible ISP for the IP block which contains the address. Look for the abuse@ address or just mail the technical contact.
Send them a polite message with the relevant log file entries (make sure to remove any private information) and ask them to take action against the offending host.
In today's internet this is quite normal sadly. There are hordes of botnets trying to login to each server they find in whole IP networks. Typically, they use simple dictionary attacks on well-known accounts (like root or certain applications accounts).
The attack targets are not found via Google or DNS entries, but the attackers just try every IP address in a certain subnet (e.g. of known root-server hosting companies). So it doesn't matter that your URL (hence the DNS entry) is rather obscure.
That's why it is so important to:
Additionally, you can install fail2ban which will scan the authlog and if it finds a certain amount of failed login attempts from an IP, it will proceed to add that IP to
/etc/hosts.deny
or iptables/netfilter in order to lock out the attacker for a few minutes.In addition to the SSH attacks, it is also becoming common to scan your webserver for vulnerable web-applications (some blogging apps, CMSs, phpmyadmin, etc.). So make sure to keep those up-to-date and securely configured too!
A few 100 is just fine... Last month I found one of my servers had 40k failed attempts. I went trough the trouble of plotting them: Map
Once I changed the ssh port and implemented Port Knocking, the number dropped to 0 :-)
I for one use a "tarpit" in addition to only allowing public key authentication and disallowing root logins.
In
netfilter
there is arecent
module, which you can use with (INPUT
chain):What that does is, that every attempt to connect to port 22 is listed by the
recent
module with IP and some other stuff under the name "tarpit" (if you're curious, look at/proc/net/xt_recent/tarpit
). Obviously you can use other names.To list or delist IPs use:
This rate-limits the attempts to 5 in 300 seconds. Please note that users with an existing connection are not bothered by that limit, because they already have an established connection and are allowed to create more (even above the rate limit).
Adjust the rules to your liking but make sure that they be added in that order (i.e. when adding then use them in this order, when inserting then in the reverse order).
This cuts down the noise immensely. It also provides actual security (against brute-forcing) unlike the perceived security of changing the port. However, I'd still recommend changing the port if it's feasible in your environment. It will cut down the noise level a lot as well ...
You can still combine this with fail2ban, although I've been running just fine without it and only the above rules.
EDIT:
It is possible to lock your self out doing this, so you can add something like the following that lets you clear you ban by knocking on a particular port:
You could implement fail2ban, or similar methods like locking SSH to your IP. Sadly bots attempt to bruteforce access all the time so it is quite normal, you need to make sure you have a good password.
Yes. It's quite normal nowadays.
Please use only public key authentication for administrative purposes if possible. Generate a private key on you workstation:
Copypaste the contents of ~/.ssh/id_dsa.pub to you servers ~/.ssh/authorized_keys (and /root/.ssh/authorized_keys, should you require direct root login).
Configure your servers /etc/ssh/sshd_config to only accept public key authentication:
If you have too many servers, you can use Puppet to run public keys and configurations to them.
Look into Denyhosts and fail2ban to block repeated SSH login attempts and see Snort if you require full IDS/IPS.
use http://denyhosts.sourceforge.net/
and yes, you should use public-key authentication and disable password auth.
The attempts are mechanized, so the numbers seem OK (yes they are high compared to some sites and the low compared to others). You should take the steps you normally have to: You consider your sites as attack targets every day, even when you do not detect an attack; not detecting an attack, does not mean it does not exist.
Yes,
this is common, but that doesn't mean you shouldn't fight the good fight. Here are some steps on how you can make your server more secure.
Avoid DNS associated IP addresses
You can greatly reduce this number in shared or colocation environments by disabling SSH access on any IP addresses associated with domain names. Unlisted non-domain IP addresses will receive less of this type of traffic, so buy an unlisted IP and only use this IP for SSH access.
Use a VPN for all SSH access
If you are in an environment in which you can implement IPsec/VPN to a private network within your server environment, this is ideal. Disable all SSH Internet access, make sure you have an integrated lights out solution. Setup your VPN, and only allow SSH access from your VPN.
Implement IP address rules for SSH access
If the VLAN isn't an option configure your router, or firewall rules to only allow SSH connections from a known IP address range.
If you follow these steps you will sleep much easier in the night knowing someone would have to compromise your hosting companies network to gain access to the server through SSH.
I'd say only getting only 500 is kind of low.
At a previous employer, one of the computer security researchers called the constant stream of break-in attempts the "internet equivalent of cosmic noise". He described it as a normal, continuous flow of malicious traffic that was searching for systems on the internet and automatically exploit scripts to attempt to hijack the system. Bot-nets and other malicious systems would perpetually scan and rescan the internet for vulnerable systems much like SETI.
In addition to using an automated lockout mechanism like fail2ban you have one more option: actually contact the abuse address ISP of the attacker. It may seem completely futile but in the case of the script-kiddie their ISP is more than willing to take action on them.
To find the abuse address, start with arin.net and lookup the IP address using whois. You may be redirected to another regional registry but eventually you can find the responsible ISP for the IP block which contains the address. Look for the abuse@ address or just mail the technical contact.
Send them a polite message with the relevant log file entries (make sure to remove any private information) and ask them to take action against the offending host.