Home / server / Questions / 1135810
Accepted
Michael Martinez
Asked: 2023-07-06 08:26:58 +0800 CST

Kubernetes: How/where to log when Secrets are deleted?

  • 772

Does kubernetes capture - in the event log or elsewhere - when Secrets are deleted? I'm not seeing this showing up in the event log: kubectl get events --field-selector involvedObject.kind=Secret --watch and I'm not finding it in /var/log/pods/kube_system-kube-apiserver/either.

We're using something called external-secrets-operator to manage secrets, and we would like to be informed when a secret is deleted, as this isn't supposed to happen with ESO, so it's an edge case that we'd like to monitor for.

linux

1 Answers

  1. Best Answer

    Using event log kubectl get events --field-selector involvedObject.kind=Secret --watch you will get existing secrets not the deleted resources, for deleted resources you need to use Audit logs, audit logs are automatically enabled for some managed kubernetes like GKE and AWS. Managed Kubernetes services like EKS, AKS or GCP provide support by routing cluster audit logs into centralized logging services (like AWS CloudWatch,GCP logs explorer) easily.

    To find out who deleted the namespace and at what time, it only gives the IAM user, below query is used to get when and who deleted the secret.

    logName="projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"
resource.type="k8s_cluster"
protoPayload.methodName:"io.k8s.core.v1.secrets"
NOT protoPayload.methodName:"get"
NOT protoPayload.methodName:"list"
NOT protoPayload.methodName:"watch"

    If you are not using any managed kubernetes, follow this blog written by Daniel Olaogun which elaborated on how to use the audit logs.

    • 1