Home / server / Questions / 329755
In Process
Draiken
Asked: 2011-11-11 07:01:20 +0800 CST

Connections stuck on SYN_RECV

  • 772

I've read tons of posts about this, but none of them have an actual answer.

We have 3 machines, one (not in our care) balance loads to the other two. The first one is working correctly with a FreeBSD, the other one was recently formatted and is now using Ubuntu Server.

The second machine is currently leaving all connections on SYN_RECV for no apparent reason. Both machines have no firewall.

dmesg shows the Possible SYN Flood attack but we know it is not the case.

What could be wrong? Is there some kernel configuration I must do? Does Ubuntu have some known issue with this?

Thanks

Edit: I found a rule on the PF of the BSD machine, I'm not sure but it should be related to this problem.

pass in log on bce1 proto tcp from <nois> to any port = http flags S/SA keep state (max 2000, source-track rule, max-src-states 120, max-src-conn 80, adaptive.start 1200, adaptive.end 2400)

It basically keepts state of SYN SYNACK flag packages... does anyone know how to translate this to IPTABLES ?

ubuntu networking

2 Answers

  1. Basically, what's happening is that the Linux box is receiving SYNs (attempting to open up a TCP connection) and then it's not getting ACKs (completely the process). The half-open connections accumulate until the server decides it's under attack.

    There are a few possible causes:

    First, the load balancer might be configured to do this. It might pass a SYN on to both boxes and only allow one connection to complete to the client, since it only needs one. The balancer might assume that ignoring the reply is harmless, since packets drop all the time. However, in this case, there's really no reason you should be having an actual problem. Make sure you have syncookies enabled, just to be sure it wouldn't cause a problem if this is the case.

    Second, the Linux box's ACK in response to the SYN might not be getting through the Linux box's firewall. This would require a pretty badly broken firewall configuration, so it's not likely.

    The less likely possibilities are:

    Third, the load balancer may reject the Linux box's ACK for some reason.

    Fourth, the load balancer may fail to transmit an ACK back to the Linux box.

    Fifth, the Linux box's firewall may drop the incoming ACK.

    • 1

  2. From what you say it sounds more than likely that the loadbalancer checks the returned SYN/ACK from the backend server to see if it is up, and only then communicating said SYN/ACK back to the client; this means in addition to loadbalancing it is also performing failover duties.

    Apparently, FreeBSD handles this differently than Ubuntu does, and the Ubuntu box sees this as a SYN flood attack.

    Sniffing the traffic from the clients and between the load balancer and the backends will tell you exactly what is happening.

    • 0