I have a kubernetes cluster running w/ 4 nodes, my main master node plus for other nodes. I used kubespray to get everything running. This is on my home network, which I will explain down bellow. What I am trying to accomplish is to get some sort of DNS resolution working so I can proceed to setup ingress.
My Setup:
- Network - Unifi setup with USG, Cloud Key, and a switch. DHCP DNS points to the Master box's internal IP plus 8.8.8.8 and 8.8.4.4, with the default search domain set to one of my personal domains (mydomain.com is used as an example)
- Master: Ubuntu 18.04 LTS running DNS (Bind9) plus serves as master for the Kubernetes cluster. He resolves internally as server.mydomain.com
- Nodes 1-3: Ubuntu 18.04 LTS serving only as Kubernetes hosts. The resolve internally as nodeX.mydomain.com, where X is the node number 1-3
I am guessing that I somehow need to tell the DNS server running on my master that anything XXX.server.mydomain.com. At least, that how I picture it should work? I am not sure, maybe I am misunderstanding it.
The thing is, even if I try to nslookup the services manually, it still doesn't seem to work. I tried something like:
> nslookup nzbget.server.mydomain.com 10.233.0.3
Server: 10.233.0.3
Address: 10.233.0.3#53
** server can't find nzbget.server.mydomain.com: NXDOMAIN
as well as
> nslookup nzbget 10.233.0.3
Server: 10.233.0.3
Address: 10.233.0.3#53
** server can't find nzbget.mydomain.com: REFUSED
With 10.233.0.3 being the IP of the CoreDNS service IP in the cluster.
I know this is just me not knowing what I am doing, but I don't even know enough to google the right things at this point. Thanks for any guidance you can offer.
You need to setup a wildcard domain/subdomain in bind9, if you search on google for "bind wildcard record" you should be find this post Wildcard DNS with BIND
I will leave @DarkVex's answer as the correct one since it did answer the original question, but wanted to add some details of my final solution for those that might stumble here in the future.
Basically, I was trying to get to my Kubernetes services via simplified internal DNS names, like jenkins.mydomain.com. With Docker alone I was creating virtual nics on the Linux host, then mapping IP/Port of the Docker containers to those IP/Ports and then pointing the DNS A record in Bind9 to that IP. I was trying to do something similar using Kubernetes, but wasn't sure how to accomplish that.
Turns out it's actually easier than I thought. First thing I had to do is add a route to my network setup to route the IP subnet used in Kubernetes to the master node's IP. So ,for instance my internal network has 3 virtual LANs: 10.0.x.x, 10.1.x.x and 10.2.x.x and my Kubernetes chose 10.233.x.x. If my master node is on 10.0.0.10, then I just told my router to send 10.233.x.x traffic to 10.0.0.10. This opened up the IPs so that I could hit them externally.
Next was to create a Service in LoadBalanced mode for the Deployments in Kubernetes. I could have also use an Ingress server, but I didn't see the need since the first step allows me to get to the Services from anywhere on my internal network. The service sits on top of all the deployed pods, exposing a single load balanced IP for all the underlying pods. The key here is that, while the pods IPs can change at any point, the service's IP stays static until you delete it.
Last step was same as before...just add an A record to Bind9 pointing to the Service's IP. Viola! I can now hit the service via a simplified hostname. The downside is that it takes a bit of manual steps to accomplish. It would have been nice to figure out how to setup the DNS to proxy all the requests down, maybe use an Ingress setup that looks at the requested hostname to route to the right services. I might do that eventually anyhow, but right now it at least gets me back to where I was with Docker.