i've set up a test machine (debian squeeze 2.6.32 on a linode 2048 machine) that interact with an api that returns large chunks of json. It calls the API 3000/minutes asynchronously, the api is returning payloads of ~450kb. There's also an http server on the box to display the calls results.
While doing netstat -s (uptime is 20 days):
254329 packets pruned from receive queue because of socket buffer overrun
50678438 packets collapsed in receive queue due to low socket buffer
This didn't sound good to me so I've followed these tutorials to tweak the TCP parameters:
http://fasterdata.es.net/fasterdata/host-tuning/linux/test-measurement-host-tuning/
and
http://www.acc.umu.se/~maswan/linux-netperf.txt
but it doesn't seems to help.
Any advice/tutorial/explaination about socket buffers that might help understanding and fixing the problem?
thanks