Kubernetes Ingress Support

Cilium uses the standard Kubernetes Ingress resource definition, with an ingressClassName of cilium. This can be used for path-based routing and for TLS termination. For backwards compatibility, the kubernetes.io/ingress.class annotation with value of cilium is also supported.

Note

The ingress controller creates a service of LoadBalancer type, so your environment will need to support this.

Cilium allows you to specify load balancer mode for the Ingress resource:

• dedicated: The Ingress controller will create a dedicated loadbalancer for the Ingress.

• shared: The Ingress controller will use a shared loadbalancer for all Ingress resources.

Each load balancer mode has its own benefits and drawbacks. The shared mode saves resources by sharing a single LoadBalancer config across all Ingress resources in the cluster, while the dedicated mode can help to avoid potential conflicts (e.g. path prefix) between resources.

Note

It is possible to change the load balancer mode for an Ingress resource. When the mode is changed, active connections to backends of the Ingress may be terminated during the reconfiguration due to a new load balancer IP address being assigned to the Ingress resource.

This is a step-by-step guide on how to enable the Ingress Controller in an existing K8s cluster with Cilium installed.

Prerequisites

• Cilium must be configured with kubeProxyReplacement as partial or strict. Please refer to kube-proxy replacement for more details.

• Cilium must be configured with the L7 proxy enabled using the --enable-l7-proxy flag (enabled by default).

• The minimum supported Kubernetes version for Ingress is 1.19.

Installation

HelmCilium CLI

Cilium Ingress Controller can be enabled with helm flag ingressController.enabled set as true. Please refer to Installation using Helm for a fresh installation.

$ helm upgrade cilium cilium/cilium --version 1.13.1 \
    --namespace kube-system \
    --reuse-values \
    --set ingressController.enabled=true \
    --set ingressController.loadbalancerMode=dedicated
$ kubectl -n kube-system rollout restart deployment/cilium-operator
$ kubectl -n kube-system rollout restart ds/cilium

If you only want to use envoy traffic management feature without Ingress support, you should only enable --enable-envoy-config flag.

$ helm upgrade cilium cilium/cilium --version 1.13.1 \
    --namespace kube-system \
    --reuse-values \
    --set-string extraConfig.enable-envoy-config=true
$ kubectl -n kube-system rollout restart deployment/cilium-operator
$ kubectl -n kube-system rollout restart ds/cilium

Additionally, the proxy load-balancing feature can be configured with the loadBalancer.l7.backend=envoy flag.

$ helm upgrade cilium cilium/cilium --version 1.13.1 \
    --namespace kube-system \
    --reuse-values \
    --set loadBalancer.l7.backend=envoy
$ kubectl -n kube-system rollout restart deployment/cilium-operator
$ kubectl -n kube-system rollout restart ds/cilium

Next you can check the status of the Cilium agent and operator:

$ cilium status

Install the latest version of the Cilium CLI. The Cilium CLI can be used to install Cilium, inspect the state of a Cilium installation, and enable/disable various features (e.g. clustermesh, Hubble).

LinuxmacOSOther

CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

Hubble CLI is also used to observe the traffic in later steps.

LinuxMacOSWindows

Download the latest hubble release:

export HUBBLE_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/hubble/master/stable.txt)
HUBBLE_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then HUBBLE_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/hubble/releases/download/$HUBBLE_VERSION/hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}
sha256sum --check hubble-linux-${HUBBLE_ARCH}.tar.gz.sha256sum
sudo tar xzvfC hubble-linux-${HUBBLE_ARCH}.tar.gz /usr/local/bin
rm hubble-linux-${HUBBLE_ARCH}.tar.gz{,.sha256sum}

Supported Ingress Annotations

Name	Description	Default Value
ingress.cilium.io/loadbalancer-mode	The loadbalancer mode for the ingress. Applicable values are dedicated and shared.	Defaults to Helm option ingressController.loadbalancerMode value.
ingress.cilium.io/service-type	The Service type for dedicated Ingress. Applicable values are LoadBalancer and NodePort.	Defaults to LoadBalancer if unspecified.
ingress.cilium.io/insecure-node-port	The NodePort to use for the HTTP Ingress. Applicable only if ingress.cilium.io/service-type is NodePort.	If unspecified, a random NodePort will be allocated by kubernetes.
ingress.cilium.io/secure-node-port	The NodePort to use for the HTTPS Ingress. Applicable only if ingress.cilium.io/service-type is NodePort.	If unspecified, a random NodePort will be allocated by kubernetes.
ingress.cilium.io/tcp-keep-alive	Enable TCP keep-alive	1 (enabled)
ingress.cilium.io/tcp-keep-alive-idle	TCP keep-alive idle time (in seconds)	10s
ingress.cilium.io/tcp-keep-alive-probe-interval	TCP keep-alive probe intervals (in seconds)	5s
ingress.cilium.io/tcp-keep-alive-probe-max-failures	TCP keep-alive probe max failures	10
ingress.cilium.io/websocket	Enable websocket	disabled

Additionally, cloud-provider specific annotations for the LoadBalancer service are supported. Please refer to the Kubernetes documentation for more details.

Examples

Please refer to one of the below examples on how to use and leverage Cilium’s Ingress features:

• Ingress HTTP Example
• Ingress gRPC Example
• Ingress Example with TLS Termination