Home / server / Questions / 1115303
Accepted
Froxz
Asked: 2022-11-10 23:54:55 +0800 CST

Ceph alertmanager config

  • 772

I've installed ceph using cephadm

Including monitoring stack prometheus, alertmanager, node-exporter

Currently I'm trying to add a telegram receiver (Telegram is supported from v0.24.0, so I've manually updated mgr/container_image_alertmanager fro 0.23 to 0.24) for alertmanager, but can't find in docs where alertmanager.yml should be created.

I can see this file is created inside ceph cluster /var/lib/ceph/{hash}/alertmanager.ceph-1/etc/alertmanager/alertmanager.yml

I've added the config to the above file as below:

templates:
  - '/etc/alertmanager/config/*.tmpl'
route:
  receiver: 'default'
  routes:
    - group_by: ['alertname']
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 30m
      receiver: 'telegram'
receivers:
- name: 'default'
  webhook_configs:
- name: 'ceph-dashboard'
  webhook_configs:
  - url: 'https://ceph-1:8443/api/prometheus_receiver'
- name: 'telegram'
  telegram_configs:
    - bot_token: <bot_token>
      chat_id: <chat_id>
      send_resolved: true
      parse_mode: 'HTML'
      api_url: 'https://api.telegram.org'
      message: '{{ template "telegram.text" . }}'

enter image description here

The receiver works fine, but after redeploying the alertmanager from Ceph dashboard the config is gone, and is logical, becuase I'm editing generated file.

Please if anyone can assist and/or point to correct direction where I should create alertmanager config to extend/override the defaults.

ceph

1 Answers

  1. Best Answer

    Reading the ceph manual regarding its monitoring, there's this part about using custom configuration for components of the stack.

    Option names

The following templates for files that will be generated by cephadm can be overridden. These are the names to be used when storing with `ceph config-key set`:
 - services/alertmanager/alertmanager.yml
 - services/grafana/ceph-dashboard.yml
 - services/grafana/grafana.ini
 - services/prometheus/prometheus.yml
 - services/prometheus/alerting/custom_alerts.yml
 - services/loki.yml
 - services/promtail.yml

You can look up the file templates that are currently used by cephadm in src/pybind/mgr/cephadm/templates:
 - services/alertmanager/alertmanager.yml.j2
 - services/grafana/ceph-dashboard.yml.j2
 - services/grafana/grafana.ini.j2
 - services/prometheus/prometheus.yml.j2
 - services/loki.yml.j2
 - services/promtail.yml.j2

    Simply take the ceph template, edit it to your liking and ceph config-key set it as the template to use when generating alert manager's configuration.

    • 1