Building Container Images

Two make targets exists to build container images automatically based on the locally checked out branch:

Developer images

Run make dev-docker-image to build a cilium-agent Docker image that contains your local changes.

ARCH=amd64 DOCKER_DEV_ACCOUNT=quay.io/myaccount DOCKER_IMAGE_TAG=jane-developer-my-fix make dev-docker-image

Run make docker-operator-generic-image (respectively, docker-operator-aws-image or docker-operator-azure-image) to build the cilium-operator Docker image:

ARCH=amd64 DOCKER_DEV_ACCOUNT=quay.io/myaccount DOCKER_IMAGE_TAG=jane-developer-my-fix make docker-operator-generic-image

The commands above assumes that your username for quay.io is myaccount.

Race detection

See section on compiling Cilium with race detection.

Official release images

Anyone can build official release images using the make target below.

DOCKER_IMAGE_TAG=v1.4.0 make docker-images-all

Experimental Docker BuildKit and Buildx support

Docker BuildKit allows build artifact caching between builds and generally results in faster builds for the developer. Support can be enabled by:

export DOCKER_BUILDKIT=1

Multi-arch image build support for arm64 (aka aarch64) and amd64 (aka x86-64) can be enabled by defining:

export DOCKER_BUILDX=1

Multi-arch images are built using a cross-compilation builder by default, which uses Go cross compilation for Go targets, and QEMU based emulation for other build steps. You can also define your own Buildx builder if you have access to both arm64 and amd64 machines. The “cross” builder will be defined and used if your current builder is “default”.

Buildx targets push images automatically, so you must also have DOCKER_REGISTRY and DOCKER_DEV_ACCOUNT defined, e.g.:

export DOCKER_REGISTRY=docker.io
export DOCKER_DEV_ACCOUNT=your-account

Currently the cilium-runtime and cilium-builder images are released for amd64 only (see the table below). This means that you have to build your own cilium-runtime and cilium-builder images:

make -C images runtime-image

After the build finishes update the runtime image references in other Dockerfiles (docker buildx imagetools inspect is useful for finding image information). Then proceed to build the cilium-builder:

make -C images builder-image

After the build finishes update the main Cilium Dockerfile with the new builder reference, then proceed to build Hubble from github.com/cilium/hubble. Hubble builds via buildx QEMU based emulation, unless you have an ARM machine added to your buildx builder:

export IMAGE_REPOSITORY=${DOCKER_REGISTRY}/${DOCKER_DEV_ACCOUNT}/hubble
CONTAINER_ENGINE="docker buildx" DOCKER_FLAGS="--push --platform=linux/arm64,linux/amd64" make image

Update the main Cilium Dockerfile with the new Hubble reference and build the multi-arch versions of the Cilium images:

make docker-images-all

Official Cilium repositories

The following table contains the main container image repositories managed by Cilium team. It is planned to convert the build process for all images based on GH actions.

Github Repository

Dockerfile

container image repository

Architectures

Build process

amd64

aarch64

github.com/cilium/cilium

images/runtime/Dockerfile

quay.io/cilium/cilium-runtime

Y

Y

GH Action

images/builder/Dockerfile

quay.io/cilium/cilium-builder

Y

Y

GH Action

images/cilium/Dockerfile

[docker|quay].io/cilium/cilium

Y

Y

GH Action

images/cilium-docker-plugin/Dockerfile

[docker|quay].io/cilium/docker-plugin

Y

Y

GH Action

images/hubble-relay/Dockerfile

[docker|quay].io/cilium/hubble-relay

Y

Y

GH Action

images/operator/Dockerfile

[docker|quay].io/cilium/operator

Y

Y

GH Action

images/operator-aws/Dockerfile

[docker|quay].io/cilium/operator-aws

Y

Y

GH Action

images/operator-azure/Dockerfile

[docker|quay].io/cilium/operator-azure

Y

Y

GH Action

images/operator-generic/Dockerfile

[docker|quay].io/cilium/operator-generic

Y

Y

GH Action

images/clustermesh-apiserver/Dockerfile

[docker|quay].io/cilium/clustermesh-apiserver

Y

Y

GH Action

github.com/cilium/proxy

Dockerfile.builder

quay.io/cilium/cilium-envoy-builder

Y

Y

GH Action

Dockerfile

quay.io/cilium/cilium-envoy

Y

Y

GH Action

github.com/cilium/image-tools

images/bpftool/Dockerfile

docker.io/cilium/cilium-bpftool

Y

Y

GH Action

images/llvm/Dockerfile

docker.io/cilium/cilium-llvm

Y

Y

GH Action

images/compilers/Dockerfile

docker.io/cilium/image-compilers

Y

Y

GH Action

images/maker/Dockerfile

docker.io/cilium/image-maker

Y

Y

GH Action

images/startup-script/Dockerfile

docker.io/cilium/startup-script

Y

Y

GH Action

Image dependency:

[docker|quay].io/cilium/cilium
 depends on:
  quay.io/cilium/cilium-builder
   depends on:
    quay.io/cilium/cilium-runtime
     depends on:
      docker.io/cilium/cilium-bpftool
      docker.io/cilium/cilium-llvm
  quay.io/cilium/cilium-envoy
   depends on:
    quay.io/cilium/cilium-envoy-builder
     depends on:
      quay.io/cilium/cilium-builder
       depends on:
        quay.io/cilium/cilium-runtime
         depends on:
          docker.io/cilium/cilium-bpftool
          docker.io/cilium/cilium-llvm

Update cilium-builder and cilium-runtime images

The steps described here, starting with a commit that updates the image versions, build the necessary images and update all the appropriate locations in the Cilium codebase. Hence, before executing the following steps, the user should have such a commit (e.g., see this commit) in their local tree. After following the steps below, the result would be another commit with the image updates (e.g,. see this commit). Please keep the two commits separate to ease backporting.

If you only wish to update the packages in these images, then you can manually update the FORCE_BUILD variable in images/runtime/Dockerfile to have a different value and then proceed with the steps below.

  1. Commit your changes and create a PR in cilium/cilium.

    $ git commit -sam "images: update cilium-{runtime,builder}"
    
  2. Ping one of the members of team/build to approve the build that was created by GitHub Actions here. Note that at this step cilium-builder build failure is expected since we have yet to update the runtime digest.

  3. Wait for build to complete. If the PR was opened from an external fork the build will fail while trying to push the changes, this is expected.

  4. If the PR was opened from an external fork, run the following commands and re-push the changes to your branch. Once this is done the CI can be executed.

    $ make -C images/ update-runtime-image
    $ git commit -sam "images: update cilium-{runtime,builder}" --amend
    $ make -C images/ update-builder-image
    $ git commit -sam "images: update cilium-{runtime,builder}" --amend
    
  5. If the PR was opened from the main repository, the build will automatically generate one commit and push it to your branch with all the necessary changes across files in the repository.

  6. Run the full CI and ensure that it passes.

  7. Merge the PR.

Image Building Process

Images are automatically created by a GitHub action: build-images. This action will automatically run for any Pull Request, including Pull Requests submitted from forked repositories, and push the images into quay.io/cilium/*-ci. They will be available there for 1 week before they are removed by the ci-images-garbage-collect workflow. Once they are removed, the developer must re-push the Pull Request into GitHub so that new images are created.