Well, I've been a victim of Ddos attacks and I really can't figure out how to avoid it. On some machines it works, while on some it doesn't or probably I'm doing it wrong in some way,
Attackers are using multiple IPs to attack my game server ports that make my game-server output 1mb/s to each IP Address, these are UDP reflective attacks technically. Here is how I see to protect them,
iptables -A INPUT -p udp -m state --state NEW -m recent --set --name DDOS --rsource
iptables -A INPUT -p udp -m state --state NEW -m recent --update --seconds 1 --hitcount 5 --name DDOS --rsource -j DROP
Technically this would block every attacker on UDP ports. I check if the attacker is blocked using "iftop" or "tcpdump" when I see that the output to attacker's IP becomes 0, this confirms me that the protection is working.
On my CentOS 6 machine running iptables 1.4.7, I am using similar method to block it and its really weird that it works sometimes and the next hour it doesn't. After a lot of tries, I restarted my system and iptables, protection worked fine and attacker was blocked. I logged into my system today and he is attacking me again and although I restarted iptables, hes not getting blocked. I dont want to restart my system again and again to make it work so what could be the issue?
Also, some people say UDP is state-less but whatever it technically is - I've always used this command and it has worked before for me, why not now or why partially now?
UDP is indeed stateless, though iptables appears to have some special handling of it.
So, looking at your rules, it seems you are trying to block any IP that sends you more then 5 packets in a second. Issue #1 with this is your legitimate clients will be doing this, so you will end up blocking them as well.
The better fix here is to limit the number of 'getstatus' packets that your server sees. The DDoS reflection attacks rely on this, and matching based on the packet contents is pretty straightforward. This also has the benefit of not effecting your legitimate clients (who should only ever be sending a few 'getstatus' requests).