When I use dmesg to look at linux kernel messages, I see a flood of....
TCP: too many orphaned sockets
messages. My guess is this is sockets that are hanging around in TIME_WAIT after being closed that are waiting to be cleared up. I am interested to know if this would be likely to affect services running on these servers.
From the LARTC HOWTO:
The implementation of the function limiting the number of orphans is here.
The possible cause of this error is system run out of socket memory.Either you need to increase the socket memory(net.ipv4.tcp_mem) or find out the cause of memory consumption
So here in my system you can see 725376(pages)*4096=2971140096bytes/1024*1024=708 megabyte
So this 708 megabyte of memory is used by application for sending and receiving data as well as utilized by my loopback interface.If at any stage this value reached no further socket can be made until this memory is released from the application which are holding socket open which you can determine using netstat -antulp.