Home / server / Questions / 722239
Accepted
stimur
Asked: 2015-09-15 10:01:36 +0800 CST

rsyslog convert json to text on the fly

  • 772

I have application logging events in JSON format for later structured queries, etc. Now I have a task to log the same messages in plain text (CSV probably) and I really don't want to log each event twice in both formats.

Is it possible to convert within rsyslog ? I have an option of tracing the output file and piping it to converter and then log with different tag, but that seems suboptimal.

Any other ideas ?

Thanks !

rsyslog

1 Answers

  1. Best Answer

    You might want to have a look at mmjsonparse. It appears to do the sort of thing you want to do. You'll need rsyslog 6.6 or higher (7 or higher is recommended). The sample config found here explains it quite well:

    # load needed modules
module(load="imuxsock") # provides support for local system logging
module(load="imklog") # provides kernel logging support
module(load="mmjsonparse") #for parsing CEE-enhanced syslog messages

# try to parse structured logs
action(type="mmjsonparse")

# define a template to print field "foo"
template(name="justFoo" type="list") {
    property(name="$!foo")
    constant(value="\n") #we'll separate logs with a newline
}

# and now let's write the contents of field "foo" in a file
action(type="omfile"
    template="justFoo"
    file="/var/log/foo")

    You'll still need to use logger or some module of whatever language your application is written in which interacts with rsyslog to write the messages...

    # logger '@cee: {"foo":"bar"}'
# cat /var/log/foo
bar

    If you send an unstructured log, or invalid JSON, nothing will be added to the log.

    • 1