Skip to main content

Advanced metrics

The Analytics tab displays CPU, memory, disk I/O and bandwidth utilisation for your nodes over the last 24 hours. For longer-term retention, application-specific metrics or custom dashboards, install additional components in the cluster.

You can find an overview of the built-in charts under View metrics.

Prerequisites​

  • A Kubernetes cluster with the status Running and a verified connection via kubectl
  • Helm installed

When are the built-in metrics not sufficient?​

  • You need a longer period than 24 hours.
  • You want to track application-specific metrics, such as requests per second or queue lengths.
  • You need metrics per pod or namespace rather than per node.
  • You want to define alerts with your own thresholds and notification methods.

metrics-server​

The metrics-server is a prerequisite for kubectl top and for horizontal pod autoscaling based on CPU and memory utilisation. First, check whether it is already running:

kubectl get deployment metrics-server -n kube-system

If the command does not produce a result, install it:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Then check:

kubectl top nodes
kubectl top pods --all-namespaces
Hinweis

The metrics-server only provides current snapshots and does not store a history. For time series data, you will need a time series database such as Prometheus.

Prometheus and Grafana​

The kube-prometheus-stack has become the go-to solution for historical data, custom metrics and alerts. It includes Prometheus, Grafana, the Alert Manager and pre-configured dashboards.

Installation via Helm​

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace

Check the installation:

kubectl get pods -n monitoring

Accessing Grafana​

Forward the port locally:

kubectl port-forward -n monitoring service/monitoring-grafana 3000:80

Grafana can then be accessed via http://localhost:3000. You can find the initial password in the corresponding secret:

kubectl get secret -n monitoring monitoring-grafana \
  -o jsonpath="{.data.admin-password}" | base64 --decode
Warnung

Do not make Grafana and Prometheus publicly accessible via a NodePort without authentication. Instead, use port-forward or an Ingress with upstream authentication.

Planning resource requirements​

A complete monitoring stack requires a significant amount of resources – Prometheus stores data in RAM and writes continuously to the hard drive. Make sure you allow for sufficient capacity and set a limit on the retention period:

helm upgrade monitoring prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --set prometheus.prometheusSpec.retention=15d

As there is no persistent block storage available within the cluster, the data collected will be lost when the Prometheus pod is restarted. For long-term analysis, run Prometheus outside the cluster or write the data to an external service.

Custom application metrics​

If your application provides metrics in Prometheus format, collect them via a ServiceMonitor:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: meine-app
  namespace: monitoring
  labels:
    release: monitoring
spec:
  namespaceSelector:
    matchNames:
      - produktion
  selector:
    matchLabels:
      app: meine-app
  endpoints:
    - port: metrics
      interval: 30s

To do this, your application’s service must provide a named port metrics.

Last updated on