Home / server / Questions / 449521
Accepted
The Shurrican
Asked: 2012-11-17 09:36:05 +0800 CST

How can i handle huge log files?

  • 772

I am running SSD servers because I have great experiences with performance. However I have webservers which serve lots of requests and the uncompresssed logfiles of one day tend to grow out the database.

I have usual standard format Webserver log files.

Is there maby a way to compress them "on the fly"?

logging

1 Answers

  1. Best Answer

    Logrotate

    This utility can automatically inform your daemon to switch logs as it renames and compresses them. Since daily is not enough, you could also run it hourly. Even if logged hourly, you can still concatenate them by day after compression with GZip.

    Compressed filesystem

    Btrfs can be setup with transparent encryption, automatically saving space in the background. Some other filesystems (ZFS, etc.) can also accomplish this.

    Compression pipe

    Apache will log directly to a pipe and most applications will write to a name pipe (mkfifo) which you can attach to your compression program.

    • 6