Traffic Splitting Example

HTTP traffic splitting is the process of sending incoming traffic to multiple backend services, based on predefined weights or other criteria. The Cilium Gateway API includes built-in support for traffic splitting, allowing users to easily distribute incoming traffic across multiple backend services. This is very useful for canary testing or A/B scenarios.

This particular example uses the Gateway API to load balance incoming traffic to different backends, starting with the same weights before testing with a 99/1 weight distribution.

Deploy the Echo App

We will use a deployment made of echo servers.

The application will reply to the client and, in the body of the reply, will include information about the Pod receiving the original request. We will use this information to illustrate how the traffic is manipulated by the Gateway.

$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes/gateway/echo-basic.yaml

Verify the Pods are running as expected.

$ kubectl get pods
NAME                      READY   STATUS    RESTARTS   AGE
echo-1-7d88f779b-m6r46    1/1     Running   0          21s
echo-2-5bfb6668b4-n7llh   1/1     Running   0          21s

Deploy the Cilium Gateway

You can find an example Gateway and HTTPRoute definition in splitting.yaml:

---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: cilium-gw
spec:
  gatewayClassName: cilium
  listeners:
  - protocol: HTTP
    port: 80
    name: web-gw-echo
    allowedRoutes:
      namespaces:
        from: Same
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: example-route-1
spec:
  parentRefs:
  - name: cilium-gw
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /echo
    backendRefs:
    - kind: Service
      name: echo-1
      port: 8080
      weight: 50
    - kind: Service
      name: echo-2
      port: 8090
      weight: 50

Notice the even 50/50 split between the two Services.

Deploy the Gateway and the HTTPRoute:

$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes/gateway/splitting.yaml

The preceding example creates a Gateway named cilium-gw that listens on port 80. A single route is defined and includes two different backendRefs (echo-1 and echo-2) and weights associated with them.

$ kubectl get gateway cilium-gw
NAME        CLASS    ADDRESS          PROGRAMMED   AGE
cilium-gw   cilium   172.18.255.200                8s

Note

Some providers like EKS use a fully-qualified domain name rather than an IP address.

Even traffic split

Now that the Gateway is ready, you can make HTTP requests to the services.

$ GATEWAY=$(kubectl get gateway cilium-gw -o jsonpath='{.status.addresses[0].value}')
$ curl --fail -s http://$GATEWAY/echo
{
 "path": "/echo",
 "host": "172.18.255.200",
 "method": "GET",
 "proto": "HTTP/1.1",
 "headers": {
  "Accept": [
   "*/*"
  ],
  "User-Agent": [
   "curl/7.81.0"
  ],
  "X-Envoy-Internal": [
   "true"
  ],
  "X-Forwarded-For": [
   "172.18.0.1"
  ],
  "X-Forwarded-Proto": [
   "http"
  ],
  "X-Request-Id": [
   "ee152a07-2be2-4539-b74d-ebcebf912907"
  ]
 },
 "httpPort": "3000",
 "namespace": "default",
 "ingress": "",
 "service": "",
 "pod": "echo-1-7d88f779b-m6r46"
}

Notice that the reply includes the name of the Pod that received the query in the pod field. For example:

"pod": "echo-2-5bfb6668b4-2rl4t"

Repeat the command several times. You should see the reply balanced evenly across both Pods and Nodes. Verify that traffic is evenly split across multiple Pods by running a loop and counting the requests:

$ while true; do curl -s -k "http://$GATEWAY/echo" >> curlresponses.txt ;done

Stop the loop with Ctrl+C. Verify that the responses are more or less evenly distributed.

$ cat curlresponses.txt| grep -c '"pod": "echo-1'
1221
$ cat curlresponses.txt| grep -c '"pod": "echo-2'
1162

Uneven (99/1) traffic split

Update the HTTPRoute weights, either by using kubectl edit httproute or by updating the value in the original manifest before reapplying it to. For example, set 99 for echo-1 and 1 for echo-2:

backendRefs:
- kind: Service
  name: echo-1
  port: 8080
  weight: 99
- kind: Service
  name: echo-2
  port: 8090
  weight: 1

Verify that traffic is unevenly split across multiple Pods by running a loop and counting the requests:

$ while true; do curl -s -k "http://$GATEWAY/echo" >> curlresponses991.txt ;done

Stop the loop with Ctrl+C. Verify that responses are more or less evenly distributed.

$ cat curlresponses991.txt| grep -c '"pod": "echo-1'
24739
$ cat curlresponses991.txt| grep -c '"pod": "echo-2'
239