Home / server / Questions / 777599
In Process
DarkSkull
Asked: 2016-05-19 10:51:43 +0800 CST

Can't reach a Kubernetes service on a guest node

  • 772

I have tried to run the guestbook example in Kubernetes Github repository but I can't reach this service from my local host. My test enviroment consists of two virtual machines (with CentOS7) provisioned by CloudStack, with OpenShift Origin installed on it. Here it's the services list:

    [root@openshift-master amd64]# ./oc get svc
NAME              CLUSTER-IP       EXTERNAL-IP   PORT(S)                   AGE
docker-registry   172.30.39.251    <none>        5000/TCP                  1d
guestbook         172.30.55.125    nodes         3000/TCP                  56m
kubernetes        172.30.0.1       <none>        443/TCP,53/UDP,53/TCP     1d
redis-master      172.30.24.94     <none>        6379/TCP                  1h
redis-slave       172.30.132.250   <none>        6379/TCP                  1h
router            172.30.33.117    <none>        80/TCP,443/TCP,1936/TCP   1d

The service exposed is guestbook. Here is the service guestbook description:

[root@openshift-master amd64]# ./oc describe svc guestbook
Name:           guestbook
Namespace:      default
Labels:         app=guestbook
Selector:       app=guestbook
Type:           NodePort
IP:         172.30.55.125
Port:           <unset> 3000/TCP
NodePort:       <unset> 30642/TCP
Endpoints:      172.17.0.6:3000,172.17.0.7:3000,172.17.0.8:3000
Session Affinity:   None
No events.

If I do:

curl 172.30.55.125:3000

It works only from the node who host the guestbook pod, from others node in the cluster and my host machine (192.168.1.2) It doesn't work.

I opened all ports in CloudStack, otherwise I can't ssh the nodes and in the node I set this firewall rule:

firewall-cmd --permanent --zone=public --add-port=30642/tcp

30642 is the NodePort, that is mandatory to reach it from out of the cluster. Have you any idea on how to resolve? Thanks in advance.

firewall kubernetes cloudstack openshift-origin

1 Answers

  1. curl 172.17.0.6:3000 (i.e. each of the Endpoints addresses) should be usable directly from every cluster node. If it doesn't work, then the cluster network is not set up correctly. This could include any firewall or SDN that filters packets sent from one node to another.

    172.30.55.125:3000 should have an entry in the iptables list on every cluster node, maintained by the local kube-proxy daemon on each cluster node. If curling a remote endpoint works but using the service virtual ip and port fails, then it's possible that kube-proxy is not working. Check its iptables entries, its process status, and its log file.

    Finally, it's possible that the guestbook app is indeed receiving the connection, but it's then aborting or blocking while it tries a doomed reverse DNS lookup.

    • 1