Home / server / Questions / 1089688
Accepted
rvs
Asked: 2022-01-12 08:35:08 +0800 CST

Setting resource limits on kube-apiserver

  • 772

Resource Management for Pods and Containers describes how to set resource requets and limits for "regular" pods in Kubernetes. Is there a supported/reccomended way to set these limits for control plane components such as kube-apiserver?

Things I considered:

  • Modifying static manifests, e.g. in /etc/kubernetes/manifests/kube-apiserver.yaml. This could work but it will be overwritten by kubeadm during next upgrade.
  • Setting kube-reserved or system-reserved flags. This could work too however again - they are defined in just one ConfigMap (e.g. kubelet-config-1.21) and will be overwritten by kubeadm during node upgrade. The same limits will apply to control plane nodes and worker nodes and I don't want that.

I can overcome this with something like ansible but then ansible will be "figting" with kubeadm and I'd like to avoid that.

What problem am I trying to solve?

I have a small homelab kubernetes installation. I'd like to allow running regular pods on control plane node(s), but I'd like to be able to reserve some resources (primarily memory) for control plane components. I.e. I'd like to be able to set requests on things like kube-apiserver so that scheduler knows not to put any other pods (they will also have appropriate requests) in its place.

kubernetes

1 Answers

  1. Best Answer

    Is there a supported/reccomended way to set these limits for control plane components such as kube-apiserver?

    Yes, you can use kubeadm init with patches command line flag. Look at this github page. The documentation of this thing could be also interested. See also official documentation: Customizing the control plane with patches:

    Kubeadm allows you to pass a directory with patch files to InitConfiguration and JoinConfiguration on individual nodes. These patches can be used as the last customization step before the control plane component manifests are written to disk.

    You can pass this file to kubeadm init with --config <YOUR CONFIG YAML>:

    apiVersion: kubeadm.k8s.io/v1beta3
kind: InitConfiguration
patches:
  directory: /home/user/somedir

    Here's example how to set resources on kube-apiserver:

    Create kube-apiserver.yaml file in some directory (e.g. /home/user/patches) with following contents:

    spec:
  containers:
    - name: kube-apiserver
      resources:
        requests:
          memory: 512Mi
        limits:
          memory: 1024Mi

    Then use --patches flag every time during node upgrade: use kubeadm upgrade node --patches /home/user/patches/ or kubeadm upgrade apply v1.22.4 --patches /home/user/patches/

    The other option will be supply extra flags to control-plane components. For this, check this guide: Customizing the control plane with flags in ClusterConfiguration:

    The kubeadm ClusterConfiguration object exposes a way for users to override the default flags passed to control plane components such as the APIServer, ControllerManager, Scheduler and Etcd. The components are defined using the following structures:

    • apiServer
    • controllerManager
    • scheduler
    • etcd
    • 1