Home / server / Questions / 352541
In Process
Strawberry
Asked: 2012-01-23 13:53:39 +0800 CST

How should I deal with logs before they get to big?

  • 772

I found out my disk was full today, and I freed up 10% just from deleting mostly logs from multiple websites. I, personally, like to keep logs as an archive, because I may need to look back on them for whatever reason. I'm new to administrating my LAMP server, so I want to pick up some good practices. How periodically should I clean out my logs? Should I even keep them? What other things can I do periodically to clean out the caches and "temporary" files for my server to keep it from using up all the diskspace again?

log-files httpd logwatch logrotate

5 Answers

  1. There is a tool called logrotate which is designed to do exactly what you want to do by hand. Depending on how you configure it, it will e.g. once a day create a new log file and renames to old one to logfile.1, logfile.2 etc. And the oldest one gets then deleted.

    Depending on your situation you maybe want to backup the log files to another server as well.

    Here is another how-to about the tool.

    • 4

  2. Yes, use logrotate to deal with logs.

    You may also want to consider using a monitoring tool like monit to warn you of critical conditions in disk space.

    • 2

  3. Use logrotate - which should already be on your debian system by default.

    You may simply need to adjust the policies to catch logs that aren't currently being rotated, or adjust the retention policies to keep the archived copies less long.

    Check /etc/logrotate.conf and the files in /etc/logrotate.d/ - if you installed Apache via the debian package, there should already be a policy in there for it that simply needs to be pointed at your logs.

    • 1

  4. Apart from using logrotate you could also use the apache-httpd means rotatelogs. But your question aims into a different area, too.

    There are propably two reasons to keep (http) logs:

    • Statistics (and other lies)
    • Keeping track of error conditions

    For decent statistic visualization there is AWStats. A program that analyzes log-files and notifies about important things is logwatch.

    If you have both in place you can rotate your logs away after some days.

    • 1

  5. The real answer (aside from the technical how- to which there are multiple ways including logrotate), is answered by explaining why you want to keep the logs. I have multiple customers that swear they need all of their logs. They keep them for years and even back them up. To date those webserver logs(IIS and Apache) haven't been looked at. Ever....

    I would suggest using logratate and simply deleting the logs depending on a reasonable period for your sites log file growth. In general, I I think 2 weeks of log files (time wise) is a reasonable retention period. I am quite happy to lower that if the size of the files appears to warrant it vs drive size. Eg if I end up with 10GB of logs in a week I think I'm lowering the threshold.

    • 1