I have fail2ban installed to ban bruteforce attempts on the ssh password. There are business requirements for not disabling password authentication on this machine.
fail2ban was installed using the same chef cookbook that effectively bans ssh attacks on other machines. There is an ssh jail configured:
# service fail2ban status
fail2ban-server (pid 5480) is running...
WARNING 'pidfile' not defined in 'Definition'. Using default one: '/var/run/fail2ban/fail2ban.pid'
Status
|- Number of jail: 1
`- Jail list: ssh
Manually banning users works:
# fail2ban-client set ssh banip 103.41.124.46
But it doesn't appear to have banned anyone automatically:
# cat /var/log/fail2ban.log
2014-11-20 18:23:47,069 fail2ban.server [67569]: INFO Exiting Fail2ban
2014-11-20 18:44:59,202 fail2ban.server [5480]: INFO Changed logging target to /var/log/fail2ban.log for Fail2ban v0.8.14
2014-11-20 18:44:59,213 fail2ban.jail [5480]: INFO Creating new jail 'ssh'
2014-11-20 18:44:59,214 fail2ban.jail [5480]: INFO Jail 'ssh' uses poller
2014-11-20 18:44:59,249 fail2ban.jail [5480]: INFO Initiated 'polling' backend
2014-11-20 18:44:59,270 fail2ban.filter [5480]: INFO Added logfile = /var/log/secure
2014-11-20 18:44:59,271 fail2ban.filter [5480]: INFO Set maxRetry = 6
2014-11-20 18:44:59,272 fail2ban.filter [5480]: INFO Set findtime = 600
2014-11-20 18:44:59,272 fail2ban.actions[5480]: INFO Set banTime = 300
2014-11-20 18:44:59,431 fail2ban.jail [5480]: INFO Jail 'ssh' started
2014-11-21 11:09:37,447 fail2ban.actions[5480]: WARNING [ssh] Ban 103.41.124.46
2014-11-21 11:10:32,602 fail2ban.actions[5480]: WARNING [ssh] Ban 122.225.97.75
2014-11-21 11:14:37,899 fail2ban.actions[5480]: WARNING [ssh] Unban 103.41.124.46
2014-11-21 11:15:32,976 fail2ban.actions[5480]: WARNING [ssh] Unban 122.225.97.75
2014-11-21 11:30:06,295 fail2ban.comm [5480]: WARNING Command ['ban', 'ssh', '189.203.240.89'] has failed. Received Exception('Invalid command',)
2014-11-21 11:30:33,966 fail2ban.actions[5480]: WARNING [ssh] Ban 189.203.240.89
2014-11-21 11:35:34,303 fail2ban.actions[5480]: WARNING [ssh] Unban 189.203.240.89
For example, this is an attack in /var/log/messages
that should have been caught and banned:
Nov 21 07:51:32 my_hostname sshd[51074]: Failed password for root from 122.225.109.219 port 1788 ssh2
Nov 21 07:51:34 my_hostname sshd[51072]: Failed password for root from 122.225.109.219 port 58285 ssh2
Nov 21 07:51:35 my_hostname sshd[51076]: Failed password for invalid user admin from 122.225.109.219 port 2221 ssh2
Nov 21 07:51:35 my_hostname sshd[51074]: Failed password for root from 122.225.109.219 port 1788 ssh2
Nov 21 07:51:37 my_hostname sshd[51072]: Failed password for root from 122.225.109.219 port 58285 ssh2
Nov 21 07:51:37 my_hostname sshd[51074]: Failed password for root from 122.225.109.219 port 1788 ssh2
Nov 21 07:51:38 my_hostname sshd[51076]: Failed password for invalid user admin from 122.225.109.219 port 2221 ssh2
Nov 21 07:51:38 my_hostname sshd[51084]: Failed password for root from 122.225.109.219 port 3501 ssh2
Nov 21 07:51:39 my_hostname sshd[51072]: Failed password for root from 122.225.109.219 port 58285 ssh2
This is also being logged in /var/log/secure
:
Nov 25 16:06:40 cluster-122-1413591380-db sshd[75769]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:06:46 cluster-122-1413591380-db sshd[75769]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:06:48 cluster-122-1413591380-db sshd[75778]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:06:55 cluster-122-1413591380-db sshd[75778]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:06:57 cluster-122-1413591380-db sshd[75780]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:03 cluster-122-1413591380-db sshd[75780]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:05 cluster-122-1413591380-db sshd[75793]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:12 cluster-122-1413591380-db sshd[75793]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:13 cluster-122-1413591380-db sshd[75797]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:21 cluster-122-1413591380-db sshd[75797]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:22 cluster-122-1413591380-db sshd[75803]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:28 cluster-122-1413591380-db sshd[75803]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:29 cluster-122-1413591380-db sshd[75809]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:36 cluster-122-1413591380-db sshd[75809]: PAM 2 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Nov 25 16:07:38 cluster-122-1413591380-db sshd[75811]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=103.41.124.41 user=root
Here's my jail.local
:
# Fail2Ban configuration file.
#
# The configuration here inherits from /etc/fail2ban/jail.conf. Any setting
# omitted here will take it's value from that file
#
# Author: Yaroslav O. Halchenko <snip>
#
#
# The DEFAULT allows a global definition of the options. They can be overridden
# in each jail afterwards.
[DEFAULT]
# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip = 127.0.0.1/8
findtime = 600
bantime = 300
maxretry = 5
# "backend" specifies the backend used to get files modification. Available
# options are "gamin", "polling" and "auto".
# yoh: For some reason Debian shipped python-gamin didn't work as expected
# This issue left ToDo, so polling is default backend for now
backend = polling
#
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
destemail = root@localhost
#
# ACTIONS
#
# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overridden globally or per
# section within jail.local file
banaction = iptables-multiport
# email action. Since 0.8.1 upstream fail2ban uses sendmail
# MTA for the mailing. Change mta configuration parameter to mail
# if you want to revert to conventional 'mail'.
mta = sendmail
# Default protocol
protocol = tcp
# Specify chain where jumps would need to be added in iptables-* actions
chain = INPUT
#
# Action shortcuts. To be used to define action parameter
# The simplest action to take: ban only
action_ = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
# ban & send an e-mail with whois report to the destemail.
action_mw = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
%(mta)s-whois[name=%(__name__)s, dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"]
# ban & send an e-mail with whois report and relevant log lines
# to the destemail.
action_mwl = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
%(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
# Choose default action. To change, just override value of 'action' with the
# interpolation to the chosen action shortcut (e.g. action_mw, action_mwl, etc) in jail.local
# globally (section [DEFAULT]) or per specific section
action = %(action_)s
#
# JAILS
#
# Next jails can inherit from the configuration in /etc/fail2ban/jail.conf.
# Enable any defined in that file jail by including
#
# [SECTION_NAME]
# enabled = true
#
# Optionally you may override any other parameter (e.g. banaction,
# action, port, logpath, etc) in that section within jail.local
[ssh]
enabled = true
port = ssh
filter = sshd
logpath = /var/log/secure
maxretry = 6
[ssh-iptables]
enabled = false
Why isn't fail2ban working? Alternately, why didn't it ban the attacker above without my manual intervention?
The parameter
logpath
should be set to a path for a log file where the SSH attempts are going to be recorded in. So if that's/var/log/messages
, then/var/log/secure
is obviously incorrect.Change the
logpath
parameter to be the correct file.On RHEL and CentOS, authentication errors go to either /var/log/messages or /var/log secure:
By default, sshd is configured with SyslogFacility set to AUTH, which goes to /var/log/messages. If you override /etc/ssh/sshd_config as follows, it will go to /var/log/secure instead:
I'm working with machines on SoftLayer cloud, and their base image configuration changed from AUTHPRIV to AUTH sometime last year.
By default, fail2ban has the following jail in /etc/fail2ban/jail.local:
I recommend adding a second jail to /etc/fail2ban/jail.local:
Afterwards, restart fail2ban to make the second jail take effect:
An alternate approach would be to expand the sshd regex in /etc/fail2ban/filter.d/sshd.conf. There is enough information in both /var/log/secure and /var/log/messages to ban IPs. Unfortunately, fail2ban can't parse all the messages without adding alternate regex. This is left as an exercise.