Home / server / Questions / 1067795
In Process
Gonios
Asked: 2021-06-25 23:41:52 +0800 CST

How do I have to configure certmanaager when using a gitlab managed cluster?

  • 772

I use a scaleway kubernetes cluster v1.21.1 managed by gitlab.

To do this, I created a Cluster Management Project with the default template. https://docs.gitlab.com/ee/user/clusters/management_project_template.html (only ingress & certmanager enabled)

I only changed the email in the cert-manager yaml files.

When I call my testsite no SSL certificate is showing up.

Gitlab created following Pods (Logs):

certmanager-cainjector

...
I0624 15:29:56.319139       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1.autoscaling"}
I0624 15:29:56.319185       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1alpha1.internal.apiserver.k8s.io"}
I0624 15:29:56.319234       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1beta1.node.k8s.io"}
I0624 15:29:56.319294       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1beta1.scheduling.k8s.io"}
I0624 15:29:56.319369       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1beta1.snapshot.storage.k8s.io"}
I0624 15:29:56.319452       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1alpha1.certmanager.k8s.io"}
I0624 15:29:56.319509       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1.apiextensions.k8s.io"}
I0624 15:29:56.319602       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1.authentication.k8s.io"}
I0624 15:29:56.319677       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1.rbac.authorization.k8s.io"}
I0624 15:29:56.319788       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1alpha1.rbac.authorization.k8s.io"}
I0624 15:29:56.319855       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1alpha1.scheduling.k8s.io"}
I0624 15:29:56.319934       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1beta1.storage.k8s.io"}
I0624 15:29:56.319995       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1.snapshot.storage.k8s.io"}
I0624 15:29:56.320065       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1beta1.coordination.k8s.io"}
I0624 15:29:56.320124       1 controller.go:242] cert-manager/controller-runtime/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="apiservice" "request"={"Namespace":"","Name":"v1.networking.k8s.io"}
E0624 15:38:14.369342       1 leaderelection.go:359] Failed to update lock: etcdserver: request timed out

certmanager-cert-manager

...
I0624 15:02:26.334768       1 controller.go:129] cert-manager/controller/certificates "level"=0 "msg"="syncing item" "key"="kubernetes-test-27639905-production/production-auto-deploy-tls" 
E0624 15:02:26.336757       1 event.go:296] Could not construct reference to: '&v1alpha1.Certificate{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"production-auto-deploy-tls", GenerateName:"", Namespace:"kubernetes-test-27639905-production", SelfLink:"", UID:"1aeb1ed7-1788-4c8f-8845-3cf76113e85f", ResourceVersion:"1574081655", Generation:3, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63760143384, loc:(*time.Location)(0x2d04f40)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"production", "app.kubernetes.io/instance":"production", "app.kubernetes.io/managed-by":"Helm", "app.kubernetes.io/name":"production", "chart":"auto-deploy-app-2.6.0", "helm.sh/chart":"auto-deploy-app-2.6.0", "heritage":"Helm", "release":"production"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference{v1.OwnerReference{APIVersion:"extensions/v1beta1", Kind:"Ingress", Name:"production-auto-deploy", UID:"b0395b36-c947-4549-8e23-5e17eea332b5", Controller:(*bool)(0xc000722c90), BlockOwnerDeletion:(*bool)(0xc000722c91)}}, Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"jetstack-cert-manager", Operation:"Update", APIVersion:"certmanager.k8s.io/v1alpha1", Time:(*v1.Time)(0xc000aa7ce0), Fields:(*v1.Fields)(nil)}}}, Spec:v1alpha1.CertificateSpec{CommonName:"", Organization:[]string(nil), Duration:(*v1.Duration)(nil), RenewBefore:(*v1.Duration)(nil), DNSNames:[]string{"le-27639905.kub-cltest.lom.li", "hostur2-kubernetes-test.kub-cltest.lom.li"}, IPAddresses:[]string(nil), SecretName:"production-auto-deploy-tls", IssuerRef:v1alpha1.ObjectReference{Name:"letsencrypt-prod", Kind:"ClusterIssuer", Group:""}, IsCA:false, Usages:[]v1alpha1.KeyUsage(nil), ACME:(*v1alpha1.ACMECertificateConfig)(0xc000aa7d20), KeySize:0, KeyAlgorithm:"", KeyEncoding:""}, Status:v1alpha1.CertificateStatus{Conditions:[]v1alpha1.CertificateCondition{v1alpha1.CertificateCondition{Type:"Ready", Status:"False", LastTransitionTime:(*v1.Time)(0xc000aa7d80), Reason:"TemporaryCertificate", Message:"Certificate issuance in progress. Temporary certificate issued."}}, LastFailureTime:(*v1.Time)(nil), NotAfter:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Normal' 'OrderComplete' 'Order "production-auto-deploy-tls-3711733499" completed successfully'
E0624 15:02:26.374421       1 sync.go:499] cert-manager/controller/certificates/certificates "msg"="error saving certificate" "error"="resourceVersion should not be set on objects to be created"  
E0624 15:02:26.374485       1 event.go:296] Could not construct reference to: '&v1alpha1.Certificate{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"production-auto-deploy-tls", GenerateName:"", Namespace:"kubernetes-test-27639905-production", SelfLink:"", UID:"1aeb1ed7-1788-4c8f-8845-3cf76113e85f", ResourceVersion:"1574081655", Generation:3, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63760143384, loc:(*time.Location)(0x2d04f40)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"production", "app.kubernetes.io/instance":"production", "app.kubernetes.io/managed-by":"Helm", "app.kubernetes.io/name":"production", "chart":"auto-deploy-app-2.6.0", "helm.sh/chart":"auto-deploy-app-2.6.0", "heritage":"Helm", "release":"production"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference{v1.OwnerReference{APIVersion:"extensions/v1beta1", Kind:"Ingress", Name:"production-auto-deploy", UID:"b0395b36-c947-4549-8e23-5e17eea332b5", Controller:(*bool)(0xc000722c90), BlockOwnerDeletion:(*bool)(0xc000722c91)}}, Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"jetstack-cert-manager", Operation:"Update", APIVersion:"certmanager.k8s.io/v1alpha1", Time:(*v1.Time)(0xc000aa7ce0), Fields:(*v1.Fields)(nil)}}}, Spec:v1alpha1.CertificateSpec{CommonName:"", Organization:[]string(nil), Duration:(*v1.Duration)(nil), RenewBefore:(*v1.Duration)(nil), DNSNames:[]string{"le-27639905.kub-cltest.lom.li", "hostur2-kubernetes-test.kub-cltest.lom.li"}, IPAddresses:[]string(nil), SecretName:"production-auto-deploy-tls", IssuerRef:v1alpha1.ObjectReference{Name:"letsencrypt-prod", Kind:"ClusterIssuer", Group:""}, IsCA:false, Usages:[]v1alpha1.KeyUsage(nil), ACME:(*v1alpha1.ACMECertificateConfig)(0xc000aa7d20), KeySize:0, KeyAlgorithm:"", KeyEncoding:""}, Status:v1alpha1.CertificateStatus{Conditions:[]v1alpha1.CertificateCondition{v1alpha1.CertificateCondition{Type:"Ready", Status:"False", LastTransitionTime:(*v1.Time)(0xc000aa7d80), Reason:"TemporaryCertificate", Message:"Certificate issuance in progress. Temporary certificate issued."}}, LastFailureTime:(*v1.Time)(nil), NotAfter:(*v1.Time)(nil)}}' due to: 'selfLink was empty, can't make reference'. Will not report event: 'Warning' 'SaveCertError' 'Error saving TLS certificate: resourceVersion should not be set on objects to be created'
E0624 15:02:26.375859       1 controller.go:131] cert-manager/controller/certificates "msg"="re-queuing item  due to error processing" "error"="resourceVersion should not be set on objects to be created" "key"="kubernetes-test-27639905-production/production-auto-deploy-tls"

certmanager-cert-manager-webhook

...
I0624 14:57:03.846840       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:05.106212       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:11.198251       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:11.411711       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:11.475789       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:11.608012       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:11.737256       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"
I0624 14:57:11.781294       1 mutation.go:120] cert-manager "level"=0 "msg"="generated patch"  "patch"="[{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsType\"},{\"op\":\"remove\",\"path\":\"/metadata/managedFields/0/fieldsV1\"}]"

Question

Is this a configuration issue?

Do I have to change something in the cluster configuration?

kubernetes gitlab cert-manager scaleway gitlab-auto-devops

1 Answers

  1. This error message comes from the fact that you are using cert-manager v0.10.1 or below with Kubernetes 1.20 or above. The issue disappears with cert-manager v0.11.0.

    I would encourage you to use a recent version of cert-manager. I noticed that helmfile.yaml has both an old and recent version of the cert-manager chart; cert-manager-1-4 is the one you should use.

    helmfiles:
#  - path: applications/cert-manager/helmfile.yaml     # ❌ cert-manager v0.10.1
#  - path: applications/cert-manager-1-4/helmfile.yaml # ✅ cert-manager v1.4.0

    Source: https://github.com/jetstack/cert-manager/issues/3615

    • 1