I am trying to mount the following platform on Google Cloud:
2 private (native) GKE clusters, in 2 different VPCs and to give them access to the internet, each vpc has a Cloud Nat configurated.
What I need is that the 2 GKE clusters interact, but peering the VPCs, I only get communication between PODs, not between POD -> Service or POD -> internal load balancer.
Clusters:
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
Shrek01 asia-east1-a 1.16.8-gke.15 <none> g1-small 1.16.8-gke.15 3 RUNNING
Shrek02 asia-east2-a 1.15.9-gke.24 <none> g1-small 1.15.9-gke.24 3 RUNNING
vpcs:
NAME SUBNET_MODE BGP_ROUTING_MODE IPV4_RANGE GATEWAY_IPV4
Shrek01 CUSTOM REGIONAL
Shrek02 CUSTOM REGIONAL
subnets:
NAME REGION NETWORK RANGE
Shrek01 asia-east1 Shrek01 192.168.13.0/24
Shrek02 asia-east2 Shrek02 192.168.14.0/24
peerings:
NAME NETWORK PEER_PROJECT PEER_NETWORK AUTO_CREATE_ROUTES STATE STATE_DETAILS
Shrek01-Shrek01-peering Shrek01 pocprod2-2019001 Shrek02 True ACTIVE [2020-05-16T14:29:57.864-07:00]: Connected.
Shrek02-Shrek01-peering Shrek02 pocprod2-2019001 Shrek01 True ACTIVE [2020-05-16T14:29:57.864-07:00]: Connected.
firewalls rules:
- "Shrek01-peering-ingress"
{
"allowed": [
{
"IPProtocol": "all"
}
],
"creationTimestamp": "2020-05-16T16:05:14.829-07:00",
"description": "",
"direction": "INGRESS",
"disabled": false,
"id": "6807007164648771397",
"kind": "compute#firewall",
"logConfig": {
"enable": false
},
"name": "peering-ingress",
"network": "https://www.googleapis.com/compute/v1/projects/pocprod2-2019001/global/networks/Shrek01",
"priority": 1000,
"selfLink": "https://www.googleapis.com/compute/v1/projects/pocprod2-2019001/global/firewalls/peering-ingress",
"sourceRanges": [
"192.168.14.0/24",
"10.113.64.0/19",
"10.213.64.0/19"
]
}
- "Shrek02-peering-ingress"
{
"allowed": [
{
"IPProtocol": "all"
}
],
"creationTimestamp": "2020-05-16T16:24:28.545-07:00",
"description": "",
"direction": "INGRESS",
"disabled": false,
"id": "7130188648920500419",
"kind": "compute#firewall",
"logConfig": {
"enable": false
},
"name": "Shrek02-peering-ingress",
"network": "https://www.googleapis.com/compute/v1/projects/pocprod2-2019001/global/networks/Shrek02",
"priority": 1000,
"selfLink": "https://www.googleapis.com/compute/v1/projects/pocprod2-2019001/global/firewalls/Shrek02-peering-ingress",
"sourceRanges": [
"192.168.13.0/24",
"10.113.32.0/19",
"10.213.32.0/19"
]
}
k8s Shrek01 cluster:
kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.213.32.1 <none> 443/TCP 85m <none>
nginx LoadBalancer 10.213.60.14 192.168.13.7 80:32612/TCP 92s app=nginx
nginx-cip ClusterIP 10.213.34.24 <none> 80/TCP 93s app=nginx
nginx-np NodePort 10.213.35.31 <none> 80:30444/TCP 92s app=nginx
kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-64b4f9bb85-9sjcp 1/1 Running 0 3m34s 10.113.34.11 gke-Shrek01-default-pool-f9ecbfcc-dz9z <none> <none>
nginx-64b4f9bb85-l2bzd 1/1 Running 0 3m34s 10.113.32.5 gke-Shrek01-default-pool-f9ecbfcc-pdll <none> <none>
nginx-64b4f9bb85-xd7kw 1/1 Running 0 3m34s 10.113.33.9 gke-Shrek01-default-pool-f9ecbfcc-v67d <none> <none>
kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
gke-Shrek01-default-pool-f9ecbfcc-dz9z Ready <none> 89m v1.16.8-gke.15 192.168.13.4 Container-Optimized OS from Google 4.19.109+ docker://19.3.1
gke-Shrek01-default-pool-f9ecbfcc-pdll Ready <none> 89m v1.16.8-gke.15 192.168.13.2 Container-Optimized OS from Google 4.19.109+ docker://19.3.1
gke-Shrek01-default-pool-f9ecbfcc-v67d Ready <none> 89m v1.16.8-gke.15 192.168.13.3 Container-Optimized OS from Google 4.19.109+ docker://19.3.1
- Checks from Shrek02's pod:
root@nginx-5c66c56f55-8jwv2:/# echo ${MY_POD_IP}
10.113.66.9
# internal load balancer
root@nginx-5c66c56f55-8jwv2:/# nc -vz 192.168.13.7 80
192.168.13.7: inverse host lookup failed: Unknown host
(UNKNOWN) [192.168.13.7] 80 (?) : Connection timed out
# intarnal load balancer's Cluster IP
root@nginx-5c66c56f55-8jwv2:/# nc -vz 10.213.60.14 80
10.213.60.14: inverse host lookup failed: Unknown host
(UNKNOWN) [10.213.60.14] 80 (?) : Connection timed out
# ClusterIP
root@nginx-5c66c56f55-8jwv2:/# nc -vz 10.213.34.24 80
10.213.34.24: inverse host lookup failed: Unknown host
(UNKNOWN) [10.213.34.24] 80 (?) : Connection timed out
# NodePort
root@nginx-5c66c56f55-8jwv2:/# nc -vz 10.213.35.31 80
10.213.35.31: inverse host lookup failed: Unknown host
(UNKNOWN) [10.213.35.31] 80 (?) : Connection timed out
# Pod IP
root@nginx-5c66c56f55-8jwv2:/# nc -vz 10.113.34.11 80
10.113.34.11: inverse host lookup failed: Unknown host
(UNKNOWN) [10.113.34.11] 80 (?) open
root@nginx-5c66c56f55-8jwv2:/# nc -vz 10.113.32.5 80
10.113.32.5: inverse host lookup failed: Unknown host
(UNKNOWN) [10.113.32.5] 80 (?) open
root@nginx-5c66c56f55-8jwv2:/# nc -vz 10.113.33.9 80
10.113.33.9: inverse host lookup failed: Unknown host
(UNKNOWN) [10.113.33.9] 80 (?) open
Did I forget any step ? I don't find the mistake.