Load-balancing & Service Discovery
This tutorial will guide you to perform load-balancing and service discovery across multiple Kubernetes clusters when using Cilium.
Prerequisites
You need to have a functioning Cluster Mesh setup, please follow the guide Setting up Cluster Mesh to set it up.
Load-balancing with Global Services
Establishing load-balancing between clusters is achieved by defining a
Kubernetes service with identical name and namespace in each cluster and adding
the annotation service.cilium.io/global: "true"
to declare it global.
Cilium will automatically perform load-balancing to pods in both clusters.
apiVersion: v1
kind: Service
metadata:
name: rebel-base
annotations:
service.cilium.io/global: "true"
spec:
type: ClusterIP
ports:
- port: 80
selector:
name: rebel-base
Disabling Global Service Sharing
By default, a Global Service will load-balance across backends in multiple clusters.
This implicitly configures service.cilium.io/shared: "true"
. To prevent service
backends from being shared to other clusters, this option should be disabled.
Below example will expose remote endpoint without sharing local endpoints.
apiVersion: v1
kind: Service
metadata:
name: rebel-base
annotations:
service.cilium.io/global: "true"
service.cilium.io/shared: "false"
spec:
type: ClusterIP
ports:
- port: 80
selector:
name: rebel-base
Deploying a Simple Example Service
In cluster 1, deploy:
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.14.2/examples/kubernetes/clustermesh/global-service-example/cluster1.yaml
In cluster 2, deploy:
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/1.14.2/examples/kubernetes/clustermesh/global-service-example/cluster2.yaml
From either cluster, access the global service:
kubectl exec -ti deployment/x-wing -- curl rebel-base
You will see replies from pods in both clusters.
In cluster 1, add
service.cilium.io/shared="false"
to existing global servicekubectl annotate service rebel-base service.cilium.io/shared="false" --overwrite
From cluster 1, access the global service one more time:
kubectl exec -ti deployment/x-wing -- curl rebel-base
You will still see replies from pods in both clusters.
From cluster 2, access the global service again:
kubectl exec -ti deployment/x-wing -- curl rebel-base
You will see replies from pods only from cluster 2, as the global service in cluster 1 is no longer shared.
In cluster 1, remove
service.cilium.io/shared
annotation of existing global servicekubectl annotate service rebel-base service.cilium.io/shared-
From either cluster, access the global service:
kubectl exec -ti deployment/x-wing -- curl rebel-base
You will see replies from pods in both clusters again.