Chris G.'s questions

Installed Calico on a K8s HA cluster

Note: same installation worked on single node installation, removing taint. But in a HA Cluster it does only have taint: node.kubernetes.io/not-ready:NoSchedule

  kubectl create -f https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml
  curl https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml -O
  kubectl create -f custom-resources.yaml

On worker node I have:

kubectl describe nodes <worker node> | grep -i taint
Taints:             node.kubernetes.io/not-ready:NoSchedule

I then did:

kubectl taint nodes <worker node> node.kubernetes.io/not-ready:NoSchedule-

This gets me from pending to ContainerCreating - problem is that it never finishes creating?

kube-system       coredns-6d4b75cb6d-mlshg           0/1     ContainerCreating   0          35m

On worker node - describe pod:

kubectl describe pod coredns-6d4b75cb6d-89ggn -n kube-system
Name:                 coredns-6d4b75cb6d-89ggn
Namespace:            kube-system
Priority:             2000000000
Priority Class Name:  system-cluster-critical
Node:                 master01worker01/192.168.64.7
Start Time:           Thu, 18 Aug 2022 14:47:22 +0200
Labels:               k8s-app=kube-dns
                      pod-template-hash=6d4b75cb6d
Annotations:          <none>
Status:               Pending
IP:                   
IPs:                  <none>
Controlled By:        ReplicaSet/coredns-6d4b75cb6d
Containers:
  coredns:
    Container ID:  
    Image:         k8s.gcr.io/coredns/coredns:v1.8.6
    Image ID:      
    Ports:         53/UDP, 53/TCP, 9153/TCP
    Host Ports:    0/UDP, 0/TCP, 0/TCP
    Args:
      -conf
      /etc/coredns/Corefile
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Limits:
      memory:  170Mi
    Requests:
      cpu:        100m
      memory:     70Mi
    Liveness:     http-get http://:8080/health delay=60s timeout=5s period=10s #success=1 #failure=5
    Readiness:    http-get http://:8181/ready delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:
      /etc/coredns from config-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tbb27 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      coredns
    Optional:  false
  kube-api-access-tbb27:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 CriticalAddonsOnly op=Exists
                             node-role.kubernetes.io/control-plane:NoSchedule
                             node-role.kubernetes.io/master:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age                   From               Message
  ----     ------            ----                  ----               -------
  Warning  FailedScheduling  2m57s                 default-scheduler  0/1 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
  Normal   Scheduled         2m15s                 default-scheduler  Successfully assigned kube-system/coredns-6d4b75cb6d-89ggn to master01worker01
  Warning  NetworkNotReady   98s (x18 over 2m10s)  kubelet            network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
  Warning  FailedMount       98s (x7 over 2m10s)   kubelet            MountVolume.SetUp failed for volume "config-volume" : object "kube-system"/"coredns" not registered

if I delete the pods, I get:

Warning FailedScheduling 76s default-scheduler 0/2 nodes are available: 2 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling

And then again if I try to un-taint using:

kubectl taint nodes <conrtrol plane> node.kubernetes.io/not-ready:NoSchedule-

Then I am back to Events:

  Type     Reason            Age              From               Message
  ----     ------            ----             ----               -------
  Warning  FailedScheduling  4m14s            default-scheduler  0/2 nodes are available: 2 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.
  Normal   Scheduled         3s               default-scheduler  Successfully assigned kube-system/coredns-6d4b75cb6d-gglk2 to master01worker01
  Warning  NetworkNotReady   1s (x2 over 3s)  kubelet            network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
  Warning  FailedMount       1s (x3 over 2s)  kubelet            MountVolume.SetUp failed for volume "config-volume" : object "kube-system"/"coredns" not registered

I have created a services that should listen on port 50051, but is is already in use.

So I need to find and kill the listener?

Step 1:

sudo lsof -i tcp:50051                                                                                                                                                                                                  
Password:
COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
launchd   1 root   43u  IPv6 0x9cf985f50f0e6545      0t0  TCP *:50051 (LISTEN)
launchd   1 root   44u  IPv4 0x9cf985f50cb5056d      0t0  TCP *:50051 (LISTEN)

Step 2:

find ~/Library/LaunchAgents /Library/LaunchAgents /Library/LaunchDaemons /System/Library/LaunchAgents /System/Library/LaunchDaemons -name "*.plist" -exec defaults read "{}" 2>/dev/null \; | grep --with-filename 50051
(standard input):            SockServiceName = 50051;

Step 3: ?

Note: I am unable to kill the listener as it is started by launchd.

lsof -ti tcp:500051 | xargs kill

If you create a localhost cloud using juju, running kubernetes - is there a way to migrate to "public" cloud?

I am thinking about using multiple microk8s nodes on MacOs M1. If I am using LXD I can have multiple VMs running microk8s in a virtuel cluster simulating one-n k8s nodes.

LXD looks very good for managing vm and system containers.

I found that LXD does not work on my MacOs M1 machine and I have to use a LXD server installed using multiples. Will this server then contain every vm that again contain microk8s?

My question is, should I just use multipass(LXD server) or/and wait for Will LXD 5.0 LTS?

I have a localhost:8080 running started using

docker run --rm -it -p 8080:8080 --entrypoint /bin/sh $$(docker build --target build -q -f $(DOCKERFILE) .)

That is just for verifying the docker build, then I exit out --rm -it and the docker ps is stoped and gone as expected.

Non of the following is showing any port 8080, but I can open localhost:8080 and is shows my website?

lsof -n -i TCP:8080

netstat -anvp tcp | awk 'NR<3 || /LISTEN/'

How can I figure out what role is missing in a service account?

Like I have the following:

kubectl auth can-i create taskrun --all-namespaces --as=system:serviceaccount:default:default

no

Ok, but where to go from here :-)

Running kubeadm init phase certs apiserver --config kubeadm.yaml

Is it possible to have multiple/custom root certificate to be used for group of users/kubectl/config files?

I am asking because, I would like to give access, on a per project basis - and then afterwards remove the custom root cert - but keep the the "original" root cert for special kubectl administrators.

I have seen that you can use ssh tunneling as a first line of defence, to protect the root certificate public key. But you still need to distribute the public signed cert, even it is behind the ssh public private key. Therefore maybe there is a way around using ssh tunneling - and putting a custom cert in certificatesDir: /etc/kubernetes/pki?

kubeadm.yaml

apiServer:
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta1
certificatesDir: /etc/kubernetes/pki

I know you can use --insecure-skip-tls-verify in config file, but it seems like a bad idea.

If reading this I then understanding, that you can set the stdin to be piped to the command as parameter to a placeholder "-"

like:

cat <<EOF | printf -  
hello  
new line
EOF

But it only prints "%"?

Why is macos ignoring etc/hosts

I have seen this: /etc/hosts file being ignored

And it is not a TAB but a space between ip address and domain.

Is there something special you need to do on MacOS to use etc/hosts.

I have a local sige mydns.com in etc/hosts:

##
# Host Database
#
# localhost is used to configure the loopback interface
# when the system is booting.  Do not change this entry.
##
127.0.0.1   localhost
255.255.255.255 broadcasthost
::1             localhost
# Added by Docker Desktop
# To allow the same kube context to work on the host and the container:
127.0.0.1 kubernetes.docker.internal
172.18.0.0 *.mydns.com

When I do curl subdomain.mydns.com it goes out to the nameservers getting the Internet ip address and not the local ip address specified in the etc/hosts?