Home / server / Questions / 967629
In Process
gspoosi
Asked: 2019-05-17 07:30:07 +0800 CST

Parsing or Reformatting Logs before feeding them to Splunk or Elastic Search

  • 772

I have very complex log messages, that I want to reduce to the most important fields in order to save quota.

The log messages are multiline and there is a lot of redundant information in them. A solution is to script something to reformat these logs into oneliners before feeding them to the splunk forwarders, but I would prefer to use an already existing solution. I thought about using something like logstash or fluentd, but they don't seem to really match my usecase.

Does anybody have experience with log parsing for Splunk or ELK and knows of a good utility do this?

Thanks & Best Regards!

syslog

1 Answers

  1. This is common when dealing with Windows event logs. Many address the problem by using SEDCMD in transforms.conf to edit the events.

    Another solution is to pre-process the events with Cribl (https://www.cribl.io/).

    • 2