IPv4 addresses used for pods are typically allocated from RFC1918 private address blocks and thus, not publicly routable. Cilium will automatically masquerade the source IP address of all traffic that is leaving the cluster to the IPv4 address of the node as the node’s IP address is already routable on the network.
For IPv6 addresses masquerading is performed only when using iptables implementation mode.
This behavior can be disabled with the option
for IPv4 and
enable-ipv6-masquerade: false for IPv6 traffic leaving the host.
- Setting the routable CIDR
- The default behavior is to exclude any destination within the IP allocation
CIDR of the local node. If the pod IPs are routable across a wider network,
that network can be specified with the option:
ipv4-native-routing-cidr: 10.0.0.0/8in which case all destinations within that CIDR will not be masqueraded.
- Setting the masquerading interface
- See Implementation Modes for configuring the masquerading interfaces.
The eBPF-based implementation is the most efficient
implementation. It requires Linux kernel 4.19 and can be enabled with
bpf.masquerade=true helm option.
Masquerading can take place only on those devices which run the eBPF masquerading
program. This means that a packet sent from a pod to an outside will be masqueraded
(to an output device IPv4 address), if the output device runs the program. If not
specified, the program will be automatically attached to the devices selected by
the BPF NodePort device detection metchanism.
To manually change this, use the
devices helm option. Use
to determine which devices the program is running on:
$ kubectl exec -it -n kube-system cilium-xxxxx -- cilium status | grep Masquerading Masquerading: BPF (ip-masq-agent) [eth0, eth1] 10.0.0.0/16
From the output above, the program is running on the
The eBPF-based masquerading can masquerade packets of the following IPv4 L4 protocols:
- ICMP (only Echo request and Echo reply)
By default, all packets from a pod destined to an IP address outside of the
ipv4-native-routing-cidr range are masqueraded, except for packets destined
to other cluster nodes. The exclusion CIDR is shown in the above output of
cilium status (
When eBPF-masquerading is enabled, traffic from pods to the External IP of cluster nodes will also not be masqueraded. The eBPF implementation differs from the iptables-based masquerading on that aspect. This limitation is tracked at GitHub issue 17177.
To allow more fine-grained control, Cilium implements ip-masq-agent in eBPF which can be
enabled with the
ipMasqAgent.enabled=true helm option.
The eBPF-based ip-masq-agent supports the
masqLinkLocal options set in a configuration file. A packet sent from a pod to
a destination which belongs to any CIDR from the
nonMasqueradeCIDRs is not
going to be masqueraded. If the configuration file is empty, the agent will provision
the following non-masquerade CIDRs:
In addition, if the
masqLinkLocal is not set or set to false, then
169.254.0.0/16 is appended to the non-masquerade CIDRs list.
The agent uses Fsnotify to track updates to the configuration file, so the original
resyncInterval option is unnecessary.
The example below shows how to configure the agent via
ConfigMap and to verify it:
$ cat agent-config/config nonMasqueradeCIDRs: - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 masqLinkLocal: false $ kubectl create configmap ip-masq-agent --from-file=agent-config --namespace=kube-system $ # Wait ~60s until the ConfigMap is mounted into a cilium pod $ kubectl -n kube-system exec -ti cilium-xxxxx -- cilium bpf ipmasq list IP PREFIX/ADDRESS 10.0.0.0/8 169.254.0.0/16 172.16.0.0/12 192.168.0.0/16
eBPF based masquerading is currently not supported for IPv6 traffic.
This is the legacy implementation that will work on all kernel versions.
The default behavior will masquerade all traffic leaving on a non-Cilium
network device. This typically leads to the correct behavior. In order to
limit the network interface on which masquerading should be performed, the
egress-masquerade-interfaces: eth0 can be used.
It is possible to specify an interface prefix as well, by specifying
eth+, all interfaces matching the prefix
eth will be used for