Home / server / Questions / 955012
Accepted
ethrbunny
Asked: 2019-02-21 18:26:39 +0800 CST

rsyslog - separate local and remote logs

  • 772

I want to use rsyslog to capture events from SANs, routers and such. (This will be forwarded to kafka and ultimately elasticsearch) So far - this is working fine. I have this configured in a config file in /etc/rsyslog.d

What's not working is that all the local log traffic (from the host running rsyslog) is being forwarded as well. I need a way to send local logs to "standard" local endpoints and remote logs to kafka.

Is this possible using rsyslog?

rsyslog

1 Answers

  1. Best Answer

    Here's a start to what seems to work:

    module(load="imuxsock")  # will listen to your local syslog
module(load="omkafka")   # lets you send to Kafka

template(name="json_lines" type="list" option.json="on") {
  constant(value="{")
  constant(value="\"timestamp\":\"")
  property(name="timereported" dateFormat="rfc3339")
  constant(value="\",\"message\":\"")
  property(name="msg")
  constant(value="\",\"host\":\"")
  property(name="hostname")
  constant(value="\",\"severity\":\"")
  property(name="syslogseverity-text")
  constant(value="\",\"facility\":\"")
  property(name="syslogfacility-text")
  constant(value="\",\"syslog-tag\":\"")
  property(name="syslogtag")
  constant(value="\"}")
}

main_queue(
  queue.workerthreads="1"      # threads to work on the queue
  queue.dequeueBatchSize="100" # max number of messages to process at once
  queue.size="10000"           # max queue size
)

if $hostname != $$myhostname then {
    action(
      broker=["kafka.server:9092"]
      type="omkafka"
      topic="syslog.inbound"
      template="json_lines"
    )

    stop
}

    It certainly needs polishing but it does seem to separate external/internal syslog messages.

    • 1