Home / server / Questions / 119937
In Process
Twirrim
Asked: 2010-03-07 14:48:59 +0800 CST

iptables, blocking large numbers of IP Addresses

  • 772

I'm looking to block IP addresses in a relatively automated fashion if they look to be 'screen scraping' content from websites that we host. In the past this was achieved by some ingenious perl scripts and OpenBSD's pf. pf is great in that you can provide it nice tables of IP addresses and it will efficiently handle blocking based on them. However for various reasons (before my time) they made the decision to switch to CentOS. iptables doesn't natively provide the ability to block large numbers of addresses (I'm told it wasn't unusual to be blocking 5000+), and I'm a bit cautious over adding that many rules into an iptable.

ipt_recent would be awesome for doing this, plus it provides a lot of flexibility for just severely slowing down access, but there is a bug in the CentOS kernel that is stopping me from using it (reported, but awaiting fix).

Using ipset would entail compiling a more up-to-date version of iptables than comes with CentOS which whilst I'm perfectly capable of doing it, I'd rather not do from a patching, security and consistency perspective.

Other than those two it looks like nfblock is a reasonable alternative. Is anyone aware of other ways of achieving this? Are my concerns about several thousand IP addresses in iptables as individual rules unfounded?

firewall performance centos iptables

4 Answers

  1. iptables is the userland tool for manipulating netfilter. netfilter is the code in the kernel that handles the packet filtering. Contextually, changing the userland tool would only change your experience not the way the filtering operates.

    I have never hit a limitation in Linux with the quantity of rules specified and I've been using Linux for firewalling since the userland tool was ipfwadm. It's notable that Netfilter wasn't introduced to Linux until the 2.4.x kernel and ipfwadm was the interface for ipfw not netfilter. OpenBSD is great for a firewall if you can continue using it in your environment.

    The limitation will likely be a physical limitation based on the system resources, with a focus on the quantity of RAM available. If you encounter issues, you may have to tune Linux's max ip connection settings in the kernel. You're unlikely to encounter either of these situations on modern hardware with a modern Linux distribution.

    If you would like to discuss the finer details of netfilter, you might be better off furthering this dialog on the netfilter mailing lists, as they will be the subject matter experts.

    If this does not answer your question, please feel free to clarify and I will be happy to revise it.

    Netfilter user hits memory limitation testing netfilter limits

    • 4

  2. The best way to block large amount of IP addresses without causing to much stress on the kernel is to null route them with command:

    root@machine# ip route add blackhole 192.168.1.1/32

    iptables is not suited well for many rules.

    http://en.wikipedia.org/wiki/Null_route

    • 3

  3. I think the bug he referenced is in the netfilter code, specifically the ipt_recent module. I seems limited to 4k (32bit) and 8k (64bit).

    see:

    https://bugzilla.redhat.com/show_bug.cgi?id=571322

    • 1

  4. 5000 rules is not a problem, however, there is http://www.hipac.org/ which does do quite well at handling enormous numbers of rules.

    • 0