Home / server / Questions / 316851
Accepted
Tom
Asked: 2011-09-30 06:49:30 +0800 CST

Split large apache log file into the past day

  • 772

I have a 3GB log file, I need to extract the past 48 hours without downloading the entire 3GB file. How can I split the file up into the past 48 hours. So I can only download that single file?

I have full SSH access and I'm able to install additional tools.

logging apache-2.2 log-files

2 Answers

  1. Best Answer

    Assuming you have shell access to the server with the log file, try

    egrep '2[789]/Sep/2011' logfile.big > /tmp/logfile.small

    That will go back to the beginning of the 27th, which is a little over 48 hours, but I would expect that to be much smaller than the whole file, and it's quick to do. Don't forget to gzip the resulting file before you transfer it, that will speed things up even more.

    • 10

  2. You can make use of the Apache log parser - Analog. It works quickly because written in C. Analog allows specifying the time ranges with the command line options +F and +T which logically stand for 'FROM' and 'TO'.

    • 3