IPv4 addresses used for pods are typically allocated from RFC1918 private address blocks and thus, not publicly routable. Cilium will automatically masquerade the source IP address of all traffic that is leaving the cluster to the IPv4 address of the node as the node’s IP address is already routable on the network.
This behavior can be disabled with the option
for IPv4 and
enable-ipv6-masquerade: false for IPv6 traffic leaving the host.
- Setting the routable CIDR
The default behavior is to exclude any destination within the IP allocation CIDR of the local node. If the pod IPs are routable across a wider network, that network can be specified with the option:
ipv6-native-routing-cidr: fd00::/100for IPv6 addresses) in which case all destinations within that CIDR will not be masqueraded.
- Setting the masquerading interface
See Implementation Modes for configuring the masquerading interfaces.
IPv6 BPF masquerading is a beta feature. Please provide feedback and file a GitHub issue if you experience any problems. IPv4 BPF masquerading is production-ready.
The eBPF-based implementation is the most efficient
implementation. It requires Linux kernel 4.19 and can be enabled with
bpf.masquerade=true helm option.
Masquerading can take place only on those devices which run the eBPF masquerading
program. This means that a packet sent from a pod to an outside address will be
masqueraded (to an output device IPv4 address), if the output device runs the program.
If not specified, the program will be automatically attached to the devices selected by
the BPF NodePort device detection mechanism.
To manually change this, use the
devices helm option. Use
to determine which devices the program is running on:
$ kubectl -n kube-system exec ds/cilium -- cilium-dbg status | grep Masquerading Masquerading: BPF (ip-masq-agent) [eth0, eth1] 10.0.0.0/16
From the output above, the program is running on the
The eBPF-based masquerading can masquerade packets of the following L4 protocols:
For ICMP, support is limited to Echo request, Echo reply, and the error message “Destination unreachable, fragmentation required, and DF flag set”.
By default, all packets from a pod destined to an IP address outside of the
ipv4-native-routing-cidr range are masqueraded, except for packets destined
to other cluster nodes (as with
ipv6-native-routing-cidr for IPv6). The
preceding output shows the exclusion CIDR of
When eBPF-masquerading is enabled, traffic from pods to the External IP of cluster nodes will also not be masqueraded. The eBPF implementation differs from the iptables-based masquerading on that aspect. This limitation is tracked at GitHub issue 17177.
To allow more fine-grained control, Cilium implements ip-masq-agent in eBPF which can be
enabled with the
ipMasqAgent.enabled=true helm option.
The eBPF-based ip-masq-agent supports the
masqLinkLocalIPv6 options set in a configuration file. A packet sent from a pod to
a destination which belongs to any CIDR from the
nonMasqueradeCIDRs is not
going to be masqueraded. If the configuration file is empty, the agent will provision
the following non-masquerade CIDRs:
In addition, if the
masqLinkLocal is not set or set to false, then
169.254.0.0/16 is appended to the non-masquerade CIDRs list. For IPv6, if
masqLinkLocalIPv6 is not set or set to false,
fe80::/10 is appended.
The agent uses Fsnotify to track updates to the configuration file, so the original
resyncInterval option is unnecessary.
The example below shows how to configure the agent via ConfigMap and to verify it:
apiVersion: v1 kind: ConfigMap metadata: name: ip-masq-agent data: config: | nonMasqueradeCIDRs: - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 masqLinkLocal: true
$ kubectl create -n kube-system -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes-ip-masq-agent/rfc1918.yaml $ # Wait ~60s until the ConfigMap is propagated into the configuration file $ kubectl -n kube-system exec ds/cilium -- cilium-dbg bpf ipmasq list IP PREFIX/ADDRESS 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
Alternatively, you can pass
--set ipMasqAgent.config.masqLinkLocal=false (or with the corresponding
option, for IPv6) when installing Cilium via Helm to
ip-masq-agent as above.
eBPF-based masquerading for IPv6 is not compatible with the host firewall.
This is the legacy implementation that will work on all kernel versions.
The default behavior will masquerade all traffic leaving on a non-Cilium
network device. This typically leads to the correct behavior. In order to
limit the network interface on which masquerading should be performed, the
egress-masquerade-interfaces: eth0 can be used.
It is possible to specify an interface prefix as well, by specifying
eth+, all interfaces matching the prefix
eth will be used for
For the advanced case where the routing layer would select different source addresses
depending on the destination CIDR, the option
can be used in order to masquerade to the source addresses rather than to the
primary interface address. The latter is then only considered as a catch-all
fallback, and for the default routes. For these advanced cases the user needs
to ensure that there are no overlapping destination CIDRs as routes on the
relevant masquerading interfaces.