Home / server / Questions / 838654
Accepted
Garreth McDaid
Asked: 2017-03-17 04:30:18 +0800 CST

Logstash multiline codec for Java stacktraces

  • 772

The logstash documentation indicates that you can collapse the multiple indented lines in a Java stacktrace log entry into a single event using the multiline codec:

https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html

input {
   syslog {
       type => syslog
       port => 8514
       codec => multiline {
            pattern => "^\s"
            what => "previous"
       }
  }
}

This is based on logstash finding an indent at the start of the line and combining that with the previous line.

However, the logstash documentation is the only place where I can find a reference to this. The general user community seems to be using elaborate grok filters to achieve the same effect.

I've tried the basic indentation pattern provided by logstash, but it doesn't work. Has anyone else managed to get this working by matching the indentation pattern?

java logstash grok

1 Answers

  1. Best Answer

    Yes, though not with the syslog {} input. I've done it with the file {} input and Tomcat logs. If the stacktraces are coming into syslog with a new event on each line, and still having the usual syslog prefix of datestamp and such, reassembling these into a unitary stackdump becomes much harder. It still can be done, but requires much more extensive filters.

    1. The input codec is not multiline; in the case of an event-per-line, the multiline codec can't handle it.
    2. A Grok filter to split out the syslog message into parts, taking the SYSLOGMESSAGE part into its own field.
    3. Using the multiline {} filter on the SYSLOGMESSAGE field to reassemble your stackdump.
    4. Use one and only one filter-worker (-w flag), it's the only way to be sure the entire stacktrace is gathered.

    If at all possible, it's best to use the file {} codec on the file the stacktraces are emitted into, and use the indentation-method you've already found.

    • 0