Home / server / Questions / 757243
In Process
Dan
Asked: 2016-02-16 01:34:54 +0800 CST

F5 High Speed Logging and Logstash GROK

  • 772

Im trying to format logs straight from our F5 using HSL and Logstash.

I've copied the example configuration exactly from the site as it contains the majority of the information I wanted logstash example

My F5 is now outputting logs to my elasticsearch cluster marking the sample log data below with the tag "message:" however when I try to apply the example grok filter, which appears to be correct; into my logstash conf file i get a _grokparseerror.

Does anyone see where i'm going wrong with my filter?

SAMPLE LOG

12.123.123.254 [13/Feb/2016:16:04:13 +0000] 123.16.1.23 /SITE/SITE_SSLV2 /SITE/SITE_SSLV2_pool 123.16.1.156 443 '/' 'GET / HTTP/1.1' 200 21003 217 'https://w2.website.co.uk/Planner/Planner' 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36'

SAMPLE GROK

%{IP:clientip} \[%{HTTPDATE:timestamp}\] %{IP:virtual_ip} %{DATA:virtual_name} %{DATA:virtual_pool_name} %{DATA:server} %{NUMBER:server_port} \"%{DATA:path}\" \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response:int} %{NUMBER:bytes:int} %{NUMBER:response_ms:int} %{QS:referrer} %{QS:agent}

MY CONFIG

filter {
  if [type] == "f5-access" {
    grok {
        match => { "message" => "%{IP:clientip} \[%{HTTPDATE:timestamp}\] %{IP:virtual_ip} %{DATA:virtual_name} %{DATA:virtual_pool_name} %{DATA:server} %{NUMBER:server_port} \"%{DATA:path}\" \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response:int} %{NUMBER:bytes:int} %{NUMBER:response_ms:int} %{QS:referrer} %{QS:agent}"}
}
    geoip {
      source => "clientip"
    }
  }
}
elasticsearch

0 Answers