One of our servers has generated a 19Gb access.log in the space of about 24 hours and the server went down in this time. We'd like to implement some measures to safeguard against DDOS attacks but looking at the access log, there are hundreds of GET requests from the actual server IP address pulling cached files from mod_pagespeed.
I have been looking at this guide and specifically limiting the amount of concurrent connections from the same IP using IP tables but from what I can gather, the server itself is the main culprit of concurrent connections (frequently above 200 at a time) or the access is from lots of different IP addresses (mostly Spain and Germany).
At the time of writing this, the server IP has 225 concurrent connections yet there are only two visitors actually browsing the site (one being me). The logs just contain tons of these GET requests from Serf/1.1.0 mod_pagespeed/1.9.32.1-4238.
I've read many other SE/SF questions including the reference to the canonical and I'm bearing in mind that this may come across as opinionated/request for recommendations but I'm genuinely unsure what I need to look at in order to protect against whatever is happening on this server.
The server is running Linux CentOs 6.4 x86_64 and Apache.
Update
Since posting this question, I have carried out every step in this guide.
Have you looked at Fail2Ban? It was created to address exactly the issue you describe. You could also address this via iptables directly as described in this post