Home / server / Questions / 1048814
In Process
giomanda
Asked: 2021-01-07 06:32:47 +0800 CST

rsyslog unexpected end of file on log files with OMFileZipLevel

  • 772

OS: Centos7
rsyslog: 8.24.0

I have various hosts sending logs to my centralised rsyslog server. I use OMFileZipLevel option in my config file to compress my logs and then zcat anytime i wish to view the contents.

Since i have upgraded to rsyslog8, whenever i try to zcat one of my compressed logs i get the following error:

#   zcat srv1.example.com.log.gz
2021-01-06T04:46:11-08:00 srv1.example.com lab: test_msg

gzip: srv1.example.com.log.gz: unexpected end of file

if i stop rsyslog server and access the file then i dont get that error message. Even if i start the server i can still see the log file without that EOF message, however when my rsyslog server receives a message and writes it to the file i start getting the same message:

#   zcat srv1.example.com.log.gz
2021-01-06T05:54:22-08:00 srv1.example.com lab: test_msg

gzip: srv1.example.com.log.gz: unexpected end of file


#   systemctl stop rsyslog

#   zcat  srv1.example.com.log.gz
2021-01-06T05:54:22-08:00 srv1.example.com lab: test_msg


#   systemctl start rsyslog

#   zcat srv1.example.com.log.gz
2021-01-06T05:54:22-08:00 srv1.example.com lab: test_msg


srv1:~$ logger -p local5.info test_msg2 @my_rsyslog_server

#   zcat srv1.example.com.log.gz
2021-01-06T03:32:09-08:00 srv1.example.com lab: hab_test
2021-01-06T05:55:27-08:00 srv1.example.com lab: test_msg2

gzip: srv1.example.com.log.gz: unexpected end of file

I was able to find a mailing list where someone mentions a similar issue and this has to do with the file still been opened by rsyslog.

Thing is that i have another rsyslog server running version 5.8.10 (Centos 6) with the exact same rsyslog configuration file but i dont have such behaviour with EOF messages on my compressed logs.

Could this be a bug in rsyslog 8.24.0 ?

syslog rsyslog

1 Answers

  1. It seems a normal behaviour. I'm not sure about closeTimeout parameter because OMFileZipLevel is usually applied to services that generates a lot of output, so I've never had a file not being written for 10 minutes (or 10 seconds btw)!

    GZIP is a stream compressor, so the files are written with a header and a tail, but the tail is not written until the file is considered closed and terminated, so zcat and gunzip will complain when reaching the end of a normal gzip file that is still being written.

    Using OMFileZipLevel just tells rsyslog to compress logs to a normal gzip file, so this still applies.

    Using veryRobustZip rsyslog can construct gzip files composed of small blocks of gzip data concatenated one after another (which is gzip-compliant) and could allow the data to be extracted without errors with zcat or gunzip (but consult rsyslog help to check some details).

    Nonetheless, I've develop gztool to easily manage compressed log files produced by rsyslog, no matter the options: you can

    • continuosuly tail a .log.gz file like with tail -f for text files, using gztool -T
    • extract all data even if the process sending logs is abruptly stopped and then restarted, which lead to gzip files with unreachable data for zcat/gunzip but that gztool can flawlessly extract with gztool -p
    • extract data from any part of the gzip file without reading it all (which is compulsory for other tools)

    Just consult the examples of use of gztool.

    The question would be why it didn't happen before the upgrade: are you sure this didn't happen before with the same rsyslog configuration?

    • 1