Running Prometheus & Grafana

Install Prometheus & Grafana

This is an example deployment that includes Prometheus and Grafana in a single deployment.

The default installation contains:

  • Grafana: A visualization dashboard with Cilium Dashboard pre-loaded.
  • Prometheus: a time series database and monitoring system.
$ kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/HEAD/examples/kubernetes/addons/prometheus/monitoring-example.yaml
namespace/cilium-monitoring created
serviceaccount/prometheus-k8s created
configmap/grafana-config created
configmap/grafana-cilium-dashboard created
configmap/grafana-cilium-operator-dashboard created
configmap/grafana-hubble-dashboard created
configmap/prometheus created
clusterrole.rbac.authorization.k8s.io/prometheus unchanged
clusterrolebinding.rbac.authorization.k8s.io/prometheus unchanged
service/grafana created
service/prometheus created
deployment.apps/grafana created
deployment.apps/prometheus created

This example deployment of Prometheus and Grafana will automatically scrape the Cilium and Hubble metrics. See the Monitoring & Metrics configuration guide on how to configure a custom Prometheus instance.

Deploy Cilium and Hubble with metrics enabled

Cilium, Hubble, and Cilium Operator do not expose metrics by default. Enabling metrics for these services will open ports 9090, 9091, and 6942 respectively on all nodes of your cluster where these components are running.

The metrics for Cilium, Hubble, and Cilium Operator can all be enabled independently of each other with the following Helm values:

  • global.prometheus.enabled=true: Enables metrics for cilium-agent.
  • global.operatorPrometheus.enabled=true: Enables metrics for cilium-operator.
  • global.hubble.metrics.enabled: Enables the provided list of Hubble metrics. For Hubble metrics to work, Hubble itself needs to be enabled with global.hubble.enabled=true. See Hubble exported metrics for the list of available Hubble metrics.

Refer to Monitoring & Metrics for more details about the individual metrics.

Note

First, make sure you have Helm 3 installed.

If you have (or planning to have) Helm 2 charts (and Tiller) in the same cluster, there should be no issue as both version are mutually compatible in order to support gradual migration. Cilium chart is targeting Helm 3 (v3.0.3 and above).

Download the Cilium release tarball and change to the kubernetes install directory:

curl -LO https://github.com/cilium/cilium/archive/master.tar.gz
tar xzvf master.tar.gz
cd cilium-master/install/kubernetes

Deploy Cilium via Helm as follows to enable all metrics:

helm install cilium ./cilium \
   --namespace kube-system \
   --set global.prometheus.enabled=true \
   --set global.operatorPrometheus.enabled=true \
   --set global.hubble.enabled=true \
   --set global.hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution,icmp,http}"

Note

You can combine the above Helm options with any of the other installation guides.

How to access Grafana

Expose the port on your local machine

kubectl -n cilium-monitoring port-forward service/grafana --address 0.0.0.0 --address :: 3000:3000

Access it via your browser: http://localhost:3000

How to access Prometheus

Expose the port on your local machine

kubectl -n cilium-monitoring port-forward service/prometheus --address 0.0.0.0 --address :: 9090:9090

Access it via your browser: http://localhost:9090

Examples

Generic

../../_images/grafana_generic.png

Network

../../_images/grafana_network.png

Policy

../../_images/grafana_policy.png ../../_images/grafana_policy2.png

Endpoints

../../_images/grafana_endpoints.png

Controllers

../../_images/grafana_controllers.png

Kubernetes

../../_images/grafana_k8s.png

Hubble General Processing

../../_images/grafana_hubble_general_processing.png

Hubble Networking

../../_images/grafana_hubble_network.png ../../_images/grafana_hubble_tcp.png ../../_images/grafana_hubble_icmp.png

Hubble DNS

../../_images/grafana_hubble_dns.png

Hubble HTTP

../../_images/grafana_hubble_http.png

Hubble Network Policy

../../_images/grafana_hubble_network_policy.png