When I create a node-pool for a specific purpose, is there a way to tell Kubernetes not to put kube-system pods such as kube-proxy on those nodes (which consumes memory)? When this happens, the actual pod that is supposed to request all the memory on these nodes becomes Pending forever.
$ kubectl get pods -l app=my-app
…
my-app-2 0/2 Pending 0 2d
$ kubectl describe nodes gke-my-cluster-my-node-pool-f05270d7-7c5d
…
Non-terminated Pods: (3 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
kube-system fluentd-cloud-logging-gke-my-cluster-my-node-pool-f05270d7-7c5d 100m (5%) 0 (0%) 200Mi (1%) 200Mi (1%)
kube-system kube-dns-3263495268-hw5bl 260m (13%) 0 (0%) 110Mi (0%) 170Mi (1%)
kube-system kube-proxy-gke-my-cluster-my-node-pool-f05270d7-7c5d 100m (5%) 0 (0%) 0 (0%) 0 (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
460m (23%) 0 (0%) 310Mi (2%) 370Mi (2%)
You can use taints for this. It's a beta feature where you mark nodes with a taint and pods with a matching tolerance. Pods without a tolerance won't be scheduled onto the tainted nodes.