Installation with external etcd

This guide walks you through the steps required to set up Cilium on Kubernetes using an external etcd. Use of an external etcd provides better performance and is suitable for larger environments. If you are looking for a simple installation method to get started, refer to the section Standard Installation.

Should you encounter any issues during the installation, please refer to the Troubleshooting section and / or seek help on Slack.

Requirements

Make sure your Kubernetes environment is meeting the requirements:

  • Kubernetes >= 1.9
  • Linux kernel >= 4.9
  • Kubernetes in CNI mode
  • Running kube-dns/coredns (When using the etcd-operator installation method)
  • Mounted BPF filesystem mounted on all worker nodes
  • Enable PodCIDR allocation (--allocate-node-cidrs) in the kube-controller-manager (recommended)

Refer to the section Requirements for detailed instruction on how to prepare your Kubernetes environment.

Configure the External Etcd

When using an external kvstore, the address of the external kvstore needs to be configured in the ConfigMap. Download the base YAML for the version of Kubernetes you are using:

wget https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.13/cilium-external-etcd.yaml
wget https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.12/cilium-external-etcd.yaml
wget https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.11/cilium-external-etcd.yaml
wget https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.10/cilium-external-etcd.yaml
wget https://raw.githubusercontent.com/cilium/cilium/v1.5/examples/kubernetes/1.9/cilium-external-etcd.yaml
  1. Open cilium-external-etcd.yaml and find the cilium-config ConfigMap and edit the endpoints: to include the list of all your etcd endpoints or a service IP that will load-balance to all etcd endpoints.
etcd-config: |-
  ---
  endpoints:
  - https://etcd1.deathstar.empire:2379
  - https://etcd2.deathstar.empire:2379
  - https://etcd3.deathstar.empire:2379
  1. Create a Kubernetes secret with the root certificate authority, and client-side key and certificate of etcd:
kubectl create secret generic -n kube-system cilium-etcd-secrets \
     --from-file=etcd-client-ca.crt=ca.crt \
     --from-file=etcd-client.key=client.key \
     --from-file=etcd-client.crt=client.crt
  1. In case you are not using a TLS-enabled etcd, comment out the configuration options in the ConfigMap referring to the key locations like this:
# In case you want to use TLS in etcd, uncomment the 'ca-file' line
# and create a kubernetes secret by following the tutorial in
# https://cilium.link/etcd-config
#ca-file: '/var/lib/etcd-secrets/etcd-client-ca.crt'
#
# In case you want client to server authentication, uncomment the following
# lines and create a kubernetes secret by following the tutorial in
# https://cilium.link/etcd-config
#key-file: '/var/lib/etcd-secrets/etcd-client.key'
#cert-file: '/var/lib/etcd-secrets/etcd-client.crt'

Deploy Cilium

kubectl create -f cilium-external-etcd.yaml

Validate the Installation

Verify that Cilium pods were started on each of your worker nodes

kubectl --namespace kube-system get ds cilium
NAME            DESIRED   CURRENT   READY     NODE-SELECTOR   AGE
cilium          4         4         4         <none>          2m

kubectl -n kube-system get deployments cilium-operator
NAME              READY   UP-TO-DATE   AVAILABLE   AGE
cilium-operator   1/1     1            1           5m25s