Home / server / Questions / 772777
Accepted
John S
Asked: 2016-04-26 07:37:01 +0800 CST

Field type not being honor in Kibana/Elasticsearch

  • 772

I'm using a logstash plug-in for a Java app using logback to forward logs on to my logstash server. I've setup a filter definition as follows:

input {
  tcp {
    port => 2856
    codec => json_lines
  }
}

filter {
  mutate {
    convert => {
      "tenantId" => "integer"
      "userId" => "integer"
    }
  }
}

Logs are being forwarded on to Elasticsearch using the following config:

output {
  elasticsearch {
    hosts => ["127.0.0.1:9200"]
    user => "user"
    password => "secure"
  }
}

The index is just going to logstash-, and when I inspect the mappings in Elasticsearch, I see the following:

  "logstash-2016.04.25" : {
      "mappings" : {
         "logs" : {
            "_all" : {
               "omit_norms" : true,
               "enabled" : true
            },
            "properties" : {
               ...
               "userId" : {
                  "type" : "long"
               },
               "tenantId" : {
                  "type" : "long"
               },
               ...
            }
         }
      }
   }

So I can see that the fields are being set with an appropriate type, but they are neither analyzed nor defined as long values when I check Kibana. What am I missing?

logging elasticsearch logstash

1 Answers

  1. Best Answer

    Assuming these are new fields in a given index, you'll need to tell Kibana to refresh it's field listing.

    Reloading the Index Fields List

    When you add an index mapping, Kibana automatically scans the indices that match the pattern to display a list of the index fields. You can reload the index fields list to pick up any newly-added fields.

    Reloading the index fields list also resets Kibana’s popularity counters for the fields. The popularity counters keep track of the fields you’ve used most often within Kibana and are used to sort fields within lists.

    To reload the index fields list:

    1. Go to the Settings > Indices tab.
    2. Select an index pattern from the Index Patterns list.
    3. Click the pattern’s Reload button.
    • 1