I recently had a server downtime. I looked everywhere and the only thing I found in my log files is:
Feb 17 18:58:04 localhost kernel: possible SYN flooding on port 80. Sending cookies.
Feb 17 18:59:33 localhost kernel: possible SYN flooding on port 80. Sending cookies.
Can someone give me more information about it. WHat is it, How can I debug the cause and how can I fix the same. I also posted ipconntrack suddenly became toooo large which has another data point I found unusual, wondering if the two things is connected as they occured exactly at the same time but at different servers. One at reverse proxy and other at actual backend Varnish server)
Thanks
Syn flood is attack where in most cases attacker fakes source address of crafted packet that attempts to establish connection to your server (port 80 in this case)
If attacker generates a lot of such packages quick he can exhaust the connection pool and by doing so prevent legitimate users to connect to host.
TCP Syn cookies is method that is used to handle situation under attack better and althou it has some false negatives, many users are able to reach your service (web) while under SYN flood attack.
This is a basic DDOS attack called a syn flood attack. Basically an attacker uses a botnet to half-open TCP connections on your machine, willing up your conntrack table. You can protect against this using iptables, or even easier by using a firewall like CSF which includes these rules built-in. I wrote about this here, and now recommend ConfigServer Firewall (CSF) which has these protections built in.
Let me know if you have any specific questions about these sorts of attacks.
This is an iptables rule you can fast use
http://blog.dubbelboer.com/2012/04/09/syn-cookies.html has an excellent writeup on this. If the connections are genuine and expected, try tuning net.ipv4.tcp_max_syn_backlog and net.core.somaxconn kernel parameters and the backlog size passed to listen() call in your application.
Above Rules set is not helpful in case of SYN attacks comes from single IP address, as other legitimate traffic will block also, as limit option not considering the single ip address, so i suggest to use connlimit or hashlimit.