Sharing of fail2ban banned IPs

I once saw a system for centralizing fail2ban data on this site, and created a modified version. The database is the same, bu I changed and created some scripts.

My system have 4 components:

1. fail2ban database

   It's a MySQL database containing only one table: erp_core_fail2ban:

   CREATE TABLE IF NOT EXISTS 'erp_core_fail2ban' (
     'id' bigint(20) unsigned NOT NULL AUTO_INCREMENT,
     'hostname' varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
     'created' datetime NOT NULL,
     'name' text COLLATE utf8_unicode_ci NOT NULL,
     'protocol' varchar(16) COLLATE utf8_unicode_ci NOT NULL,
     'port' varchar(32) COLLATE utf8_unicode_ci NOT NULL,
     'ip' varchar(64) COLLATE utf8_unicode_ci NOT NULL,
     PRIMARY KEY ('id'),
     KEY 'hostname' ('hostname','ip')
   ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
   

2. fail2ban.php

   Every time a host is banned, it will populate the database:

   
   <?php
   require_once("/etc/fail2ban/phpconfig.php");
   
   $name = $_SERVER["argv"][1];
   $protocol = $_SERVER["argv"][2];
   $port = $_SERVER["argv"][3];
   if (!preg_match('/^\d{1,5}$/', $port))
       $port = getservbyname($_SERVER["argv"][3], $protocol);
   $ip = $_SERVER["argv"][4];
   
   $hostname = gethostname();
   
   $query = "INSERT INTO 'erp_core_fail2ban' set hostname='" . addslashes($hostname) . "', name='" . addslashes($name) ."', protocol='" . addslashes($protocol) . "', port='" . addslashes($port) . "', ip='" . addslashes($ip) . "', created=NOW()";
   $result = mysql_query($query) or die('Query failed: ' . mysql_error());
   mysql_close($link);
   exit;
   ?>
   

3. cron2ban

   You put this to run on crontab, every minute. It will retrieve the last added hosts, and ban them.

   
   <?php
   // phpconfig.php will have database configuration settings
   require_once("/etc/fail2ban/phpconfig.php");
   
   // file with only a line, containing the last id banned
   $lastbanfile="/etc/fail2ban/lastban";
   
   $lastban = file_get_contents($lastbanfile);
   
   // select only hosts banned after last check
   $sql = "select id, ip from erp_core_fail2ban where id > $lastban";
   $result = mysql_query($sql) or die('Query failed: ' . mysql_error());
   mysql_close($link);
   
   while ($row = mysql_fetch_array($result)) {
           //
           $id = $row['id'];
           $ip = $row['ip'];
   
       exec("fail2ban-client set $jail banip $ip");
   
   
   }
   
   // $id contains the last banned host, add it to the config file
   file_put_contents($lastbanfile, $id);
   ?>
   

4. phpconfig

   This file goes to /etc/fail2ban and have database configuration and jail selection.

   
   <?php
   // jail to be used
   $jail = "ssh";
   
   // file to keep the last ban
   $lastbanfile="/etc/fail2ban/lastban";
   
   // database configuration
   $dbserver="localhost";
   $dbuser="root";
   $dbpass="root";
   $dbname="fail2ban";
   
   // connect to database
   $link = mysql_connect($dbserver, $dbuser, $dbpass) or die('Could not connect: ' . mysql_error());
   mysql_select_db($dbname) or die('Could not select database');
   
   ?>
   

Create those files and change the configuration from fail2ban:

After the line with actionban = ..... a new row inserted to invoke the PHP script:

/root/fail2ban.php <name> <protocol> <port> <ip>

Using this structure on all your servers will assure that every time one host gets banned on one server, all the other servers will ban it too.

So I did a bunch of research on how to do this after watching the same ip address hit my cluster of web servers one after another. Since I'm using AWS I figured there might be an easy way and its working beautifully in my first two days of testing 5 servers.

First thing I recommend is temporarily disabling SELinux, we will deal with it at the end. I'm not an SELinux expert but what I did works so far.

The primary requirement is a shared file source, I use AWS EFS. Once the new drive is provisioned and mounted, I changed logtarget inside of /etc/fail2ban/fail2ban.conf to a subfolder in the EFS drive.

logtarget = /efsmount/fail2ban/server1.log

Then I wrote a simple filter and placed it in /etc/fail2ban/filter.d/fail2ban-log.conf

[Definition]

failregex = .* Ban <HOST>

ignoreregex =

Added the filter to /etc/fail2ban/jail.local

[fail2ban-log]
enabled = true
port = http,https
findtime = 86400 ; 1 day
logpath  = /efsmount/fail2ban/server1.log
        /efsmount/fail2ban/server2.log
        /efsmount/fail2ban/server3.log
        /efsmount/fail2ban/server4.log
maxretry = 1

Then restarted fail2ban

sudo fail2ban-client reload

So far so good! No the painful part is SELinux. After I let fail2ban run for a bit I ran this command that would allow fail2ban through the filters.

sudo grep fail2ban /var/log/audit/audit.log | sudo audit2allow -M fail2ban-nfs

Audit2allow will tell you to run this command

sudo semodule -i fail2ban-nfs.pp

I am still checking my SELinux logs here and there to see if there are any more denials. If anyone has a tip on how to get that clear SELinux with another method that would be awesome.

sudo cat /var/log/audit/audit.log |grep fail2ban |grep denied

At this point I was still getting errors when restarting fail2ban. There is a bug when using action=action_mwl in jail.local. After a bit of googling I found this which is working so far. From what I read its because of the line breaks in the logpath directive pointing to multiple files. I tried with commas, spaces, etc nothing else worked with action_mwl.

action_mwm = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
             %(mta)s-whois-matches[name=%(__name__)s, dest="%(destemail)s", chain="%(chain)s"]

action = %(action_mwm)s

Don't forget to turn SELinux back on!

An alternative to fail2ban is DenyHosts which comes with a synchronization functionality. Installation is fairly similar to fail2ban, see Cyberciti's tutorial for more details.

The problem is that the synchronization service is centralized and source code of the server side doesn't seem to be available, so you can't easily start your own DenyHosts service and you have to rely on 3rd party (which might be fine for some use cases).

I've just implemented this and so far it seems to be working well. However, I had to update some of the php because the scripts in the original answer use deprecated functions.

Here are the updated scripts

phpconfig.php

#!/usr/bin/php
<?php
// jail to be used
$jail = "ssh";

// file to keep the last ban
$lastbanfile="/etc/fail2ban/lastban";

// database configuration
$dbserver="[your.mysql.hostname]";
$dbport="[sql.port.default.is.3306]";
$dbuser="[sql.user";
$dbpass="[sql.password]";
$dbname="[sql.table]";

// connect to database
$link = mysqli_connect($dbserver, $dbuser, $dbpass, $dbname, $dbport) or die('Could not connect: ' . mysqli_error());
mysqli_select_db($link,$dbname) or die('Could not select database');

?>

fail2ban.php

#!/usr/bin/php 
<?php
require_once("/etc/fail2ban/phpconfig.php");

$name = $_SERVER["argv"][1];
$protocol = $_SERVER["argv"][2];
$port = $_SERVER["argv"][3];
if (!preg_match('/^\d{1,5}$/', $port))
    $port = getservbyname($_SERVER["argv"][3], $protocol);
$ip = $_SERVER["argv"][4];

$hostname = gethostname();

$query = "INSERT INTO erp_core_fail2ban (hostname,created,name,protocol,port,ip) VALUES ('$hostname',NOW(),'$name','$protocol','$port','$ip')";
echo $query;
$result = mysqli_query($link,$query) or die('Query failed: ' . mysqli_error($link));
mysqli_close($link);
exit;
?>

cron2ban.php

#!/usr/bin/php
<?php
// phpconfig.php will have database configuration settings
require_once("/etc/fail2ban/phpconfig.php");

// file with only a line, containing the last id banned
$lastbanfile="/etc/fail2ban/lastban";

$lastban = file_get_contents($lastbanfile);
// select only hosts banned after last check
$sql = "SELECT id,ip FROM erp_core_fail2ban WHERE id > $lastban";
$result = mysqli_query($link,$sql) or die('Query failed: ' . mysqli_error($link));
mysqli_close($link);

while ($row = mysqli_fetch_array($result)) {
        //
        $id = $row['id'];
        $ip = $row['ip'];

    exec("fail2ban-client set $jail banip $ip");


}

// $id contains the last banned host, add it to the config file
file_put_contents($lastbanfile, $id);
?>

Also, wherever you place the fail2ban.php action, it must be indented as much as the line above it. For example:

actionban = ...
            /etc/fail2ban/fail2ban.php

Otherwise fail2ban will not start. I hope this helps anyone trying to deploy this.

Yes and yes. Both can be done.

You need to find a suitable mechanism to share the list of IPs. If you're using AWS for example you could take advantage of s3. You could use rsync between Linux hosts, or a database common to all hosts. You could knock up a service with you favourite programming language that provides a restful API the choice is yours.

In terms if sharing the list public ally you could create a website and host a simple text file, some already provide such lists (not crowd sourced that I know of). How to create your own site/service would be outside the scope of an answer, however shouldn't be terribly difficult to do.

Is there an easy way to share banned IPs between hosts I control?

A fairly manual setup would be to change the configuration that calls iptables to update rules so that it calls a script of your own devising which loops through a list of hosts (read from a file?) and makes the iptables calls on each via SSH. You would need key-based auth between all the hosts configured for this to work. Admin automation tools like puppet may make setting this up and maintaining it easier. This would not be terribly efficient but unless you see a massive amount of probing traffic (and/or have a massive number of hosts) then I'm sure it would be good enough. If you only have a few hosts then you don't even need to loop through a file: configure each to just call the others in order. The scripting effort will be minimal.

Is there a way to share banned IPs publicly?

There are no doubt many ways. Have the script(s) above drop the data into a DB and have clients read from that, polling for new rules and running them as they come in. The simple "run a rule as you see it" won't be perfect if many hosts are submitting information, for example this case:

1. At 12:00 server 1 says "ban host X now", and "unban host X in an hour".
2. At 12:45 server 2 says "ban host X now", and "unban host X in an hour".
3. The overlap means server 3 will ban host X for an hour, not an hour+45minutes if following the instructions in order.

but this should not be a significant problem, and if you get a bit more clever with the database you could manage multiple submissions more cleanly if you decided it was worth the effort.

Running that as a public service would open you to a world of admin hassle though:

• Managing bandwidth and other resources if you gain a lot of users.
• Arranging and enforcing payment methods should you try deal with the resourcing issue by charging for access in some way.
• Dealing with attempts to pollute your database, a bad actor trying to get a competitor banned from the places subscribing to the list as a commercial inconvenience or blackmail attempt.
• Dealing with complaints when someone is banned and thinks they shouldn't be.
• Dealing with DDoS attacks which will come if your service is at all successful in inconveniencing someone's bots.