Kubernetes without kube-proxy (beta)

This guide explains how to provision a Kubernetes cluster without kube-proxy, and to use Cilium to replace it. For simplicity, we will use kubeadm to bootstrap the cluster.

For installing kubeadm and for more provisioning options please refer to the official kubeadm documentation.

Initialize the control-plane node:

kubeadm init --pod-network-cidr= --skip-phases=addon/kube-proxy

In K8s 1.15 and older it is not yet possible to disable kube-proxy via --skip-phases=addon/kube-proxy in kubeadm, therefore the below workaround for manually removing the kube-proxy DaemonSet and cleaning the corresponding iptables rules after kubeadm initialization is still necessary (kubeadm#1733).

Initialize control-plane as first step:

kubeadm init --pod-network-cidr=

Then delete the kube-proxy DaemonSet and remove its iptables rules as following:

kubectl -n kube-system delete ds kube-proxy
iptables-restore <(iptables-save | grep -v KUBE)

Afterwards, join worker nodes by specifying the control-plane node IP address and the token returned by kubeadm init:

kubeadm join <..>

Download the Cilium release tarball and change to the kubernetes install directory:

curl -LO https://github.com/cilium/cilium/archive/v1.6.tar.gz
tar xzvf v1.6.tar.gz
cd cilium-1.6/install/kubernetes

Install Helm to prepare generating the deployment artifacts based on the Helm templates.

Next, generate the required YAML files and deploy them. Replace $API_SERVER_IP and $API_SERVER_PORT with the control-plane node IP address and the kube-apiserver port number reported by kubeadm init (usually it is 6443).

helm template cilium \
    --namespace kube-system \
    --set global.nodePort.enabled=true \
    --set global.k8sServiceHost=$API_SERVER_IP \
    --set global.k8sServicePort=$API_SERVER_PORT \
> cilium.yaml
kubectl apply -f cilium.yaml

This will install Cilium as a CNI plugin with the BPF kube-proxy replacement. See Kubernetes NodePort (beta) for requirements and configuration options for NodePort services.

Finally, verify that Cilium has come up correctly on all nodes:

kubectl -n kube-system get pods -l k8s-app=cilium
NAME                READY     STATUS    RESTARTS   AGE
cilium-crf7f        1/1       Running   0          10m
cilium-mkcmb        1/1       Running   0          10m