Home / server / Questions / 1160162
Accepted
Joan
Asked: 2024-05-30 17:47:59 +0800 CST

Installation loki grafana limit resources used

  • 772

0

I'm following the guide https://grafana.com/docs/loki/latest/setup/install/helm/install-monolithic/ to install loki in my k8s cluster:

deploymentMode: SingleBinary
loki:
  commonConfig:
    replication_factor: 1
  storage:
    type: 'filesystem'
  schemaConfig:
    configs:
    - from: "2024-01-01"
      store: tsdb
      index:
        prefix: loki_index_
        period: 24h
      object_store: filesystem # we're storing on filesystem so there's no real persistence here.
      schema: v13
singleBinary:
  replicas: 1
read:
  replicas: 0
backend:
  replicas: 0
write:
  replicas: 0

The installation works "fine" but if I execute kubectl describe pod loki-chunks-cache-0. I see the following error in the loki-chunks-cache-0:

Warning FailedScheduling 59s (x297 over 24h) default-scheduler 0/2 nodes are available: 1 Too many pods, 2 Insufficient memory. preemption: 0/2 nodes are

And the following resource requests:

memcached:
    Image:      memcached:1.6.23-alpine
    Port:       11211/TCP
    Host Port:  0/TCP
    Args:
      -m 8192
      --extended=modern,track_sizes
      -I 5m
      -c 16384
      -v
      -u 11211
    Limits:
      memory:  9830Mi
    Requests:
      cpu:        500m
      memory:     9830Mi

I understand that the problem is that I've not enough memory in my k8s cluster. But is there anyway that I can indicate in my yaml the limits/requests for this cache to reduce these needs or to disable it entirely?

grafana

1 Answers

  1. Best Answer

    From official grafana/loki helm chart repository https://github.com/grafana/loki/tree/main/production/helm/loki I have found that memcached-chunks-cache StatefulSet resources are managed by this specific value in the Helm chart:

    # loki/values.yaml

chunksCache
  allocatedMemory: 8192

    Found in https://github.com/grafana/loki/blob/main/production/helm/loki/values.yaml#L2979. Furthermore it can be disabled with:

    # loki/values.yaml

chunksCache:
  enabled: false

    Found in https://github.com/grafana/loki/blob/main/production/helm/loki/values.yaml#L2965.

    Unfortunately I have found that cpu request is hardcoded (ref: https://github.com/grafana/loki/blob/main/production/helm/loki/templates/memcached/_memcached-statefulset.tpl#L101).

    This is what my values file looks like:

    # loki.yaml
...
chunksCache:
  ...
  allocatedMemory: 1024
  ...
...

    And this configuration works fine on my local cluster

    Get loki pods

    Hope that this will be useful!

    • 1