Installation with external etcd¶
This guide walks you through the steps required to set up Cilium on Kubernetes using an external etcd. Use of an external etcd provides better performance and is suitable for larger environments. If you are looking for a simple installation method to get started, refer to the section Installation with managed etcd.
When do I need to use a kvstore?¶
Unlike the section Quick Installation, this guide explains how to configure Cilium to use an external kvstore such as etcd. If you are unsure whether you need to use a kvstore at all, the following is a list of reasons when to use a kvstore:
- If you want to use the Multi-Cluster (Cluster Mesh) functionality.
- If you are running in an environment with more than 250 nodes, 5k pods, or if you observe a high overhead in state propagation caused by Kubernetes events.
- If you do not want Cilium to store state in Kubernetes custom resources (CRDs).
Make sure your Kubernetes environment is meeting the requirements:
- Kubernetes >= 1.9
- Linux kernel >= 4.9
- Kubernetes in CNI mode
- Mounted BPF filesystem mounted on all worker nodes
- Recommended: Enable PodCIDR allocation (
--allocate-node-cidrs) in the
Refer to the section Requirements for detailed instruction on how to prepare your Kubernetes environment.
You will also need an external etcd version 3.1.0 or higher
When using an external kvstore, the address of the external kvstore needs to be
configured in the ConfigMap. Download the base YAML and configure it with
First, make sure you have Helm 3 installed.
If you have (or planning to have) Helm 2 charts (and Tiller) in the same cluster, there should be no issue as both version are mutually compatible in order to support gradual migration. Cilium chart is targeting Helm 3 (v3.0.3 and above).
Setup Helm repository:
helm repo add cilium https://helm.cilium.io/
Deploy Cilium release via Helm:
helm install cilium cilium/cilium --version 1.8.2 \ --namespace kube-system \ --set global.etcd.enabled=true \ --set "global.etcd.endpoints=http://etcd-endpoint1:2379" \ --set "global.etcd.endpoints=http://etcd-endpoint2:2379"
Optional: Configure the SSL certificates¶
Create a Kubernetes secret with the root certificate authority, and client-side key and certificate of etcd:
kubectl create secret generic -n kube-system cilium-etcd-secrets \ --from-file=etcd-client-ca.crt=ca.crt \ --from-file=etcd-client.key=client.key \ --from-file=etcd-client.crt=client.crt
Adjust the helm template generation to enable SSL for etcd and use https instead of http for the etcd endpoint URLs:
helm install cilium cilium/cilium --version 1.8.2 \ --namespace kube-system \ --set global.etcd.enabled=true \ --set global.etcd.ssl=true \ --set "global.etcd.endpoints=https://etcd-endpoint1:2379" \ --set "global.etcd.endpoints=https://etcd-endpoint2:2379"
Validate the Installation¶
Verify that Cilium pods were started on each of your worker nodes
kubectl --namespace kube-system get ds cilium NAME DESIRED CURRENT READY NODE-SELECTOR AGE cilium 4 4 4 <none> 2m kubectl -n kube-system get deployments cilium-operator NAME READY UP-TO-DATE AVAILABLE AGE cilium-operator 1/1 1 1 5m25s