Kepler
Purpose:: Analyse Kubernetes power consumption and visualize it
Type:: #tool/dev
Website:: https://sustainable-computing.io/
Docs:: https://sustainable-computing.io/installation/kepler/
Source Code:: https://github.com/sustainable-computing-io/kepler
Community:: https://github.com/sustainable-computing-io/kepler/discussions
Description
Kepler = Kubernetes-based Efficient Power Level Exporter
Kepler (Kubernetes-based Efficient Power Level Exporter) is a Prometheus exporter. It uses eBPF to probe CPU performance counters and Linux kernel tracepoints.
These data and stats from cgroup and sysfs can then be fed into ML models to estimate energy consumption by Pods.
Technology Radar (ThoughtWorks)
Assess as of April 2023
Measuring energy consumption is an important step for teams to reduce the carbon footprint of their software. Cloud Carbon Footprint (CCF) estimates energy based on billing and usage data retrieved from cloud APIs. Kepler — short for Kubernetes-based Efficient Power Level Exporter — goes one step further: it uses software counters via RAPL, ACPI and nvml to measure power consumption by hardware resources and employs an eBPF-based approach to attribute power consumption to processes, containers and Kubernetes pods.
Methodology
Collecting System Power Consumption
Data sources on bare-metal (Kepler Energy Sources):
Modeling Approach
There are 3 model approaches:
- Power Ratio
- default
- only possible if total power is known
- Power Estimation
- power is estimated by using usage metrics
- Pre-trained Power Model
Usage Scenarios
Baremetal with x86 architecture and no power meter:
- Node Total Power: Power Estimation
- Node Component Powers Measurement (RAPL)
- Pod Power: Power Ratio
Per Process/Container Measurement
- eBPF is utilized to extract process-related resource utilization metrics, by default CPU instructions are used (config)
- Ratio Power Model: calculates the ratio of a process’s resource utilization to the entire system’s resource utilization and then multiplying this ratio by the dynamic power consumption of a resource
- Power consumption of processes is aggregated into containers and Kubernetes Pods levels
More information:
- Sources / Design decisions for energy splitting · sustainable-computing-io/kepler · Discussion #548 · GitHub
See also Measure energy consumption of software per process.
Isolating Idle Power
Sunyanan Choochotkaew, Marcelo Amaral, Huamin Chen - – Idle Power Matters- Kepler Metrics for Pub...
Installation
Using Helm
GitHub - sustainable-computing-io/kepler-helm-chart
Add helm repo:
helm repo add kepler https://sustainable-computing-io.github.io/kepler-helm-chart
Install:
helm install kepler kepler/kepler --namespace kepler --create-namespace
Uninstall:
helm delete kepler --namespace kepler
Estimation
If Kepler doesn't have access to interfaces like RAPL, it uses model-based estimation approach.
In the most minimal deployment, a Local Linear Regression Estimator is used that uses usage metrics to estimate power consumption. No special configuration or extra deployment is required.
If you want to get a model with better estimation accuracy, Kepler may connect to a remote Kepler Model Server.
More Information: Kepler Power Estimation Deployment
Models
Model Version 0.7: https://github.com/sustainable-computing-io/kepler-model-db/tree/main/models/v0.7
Model Server
GitHub - sustainable-computing-io/kepler-model-server: Model Server for Kepler
🔗 References
Articles:
- 2023-02-01: Kepler project aims to help curb Kubernetes energy waste | TechTarget
- 2023-10-11: Exploring Kepler’s potentials: unveiling cloud application power consumption | CNCF
Podcasts:
Research Papers:
- Amaral, M., Chen, H., Chiba, T., Nakazawa, R., Choochotkaew, S., Lee, E. K., & Eilam, T. (2023). Kepler: A Framework to Calculate the Energy Consumption of Containerized Applications. 2023 IEEE 16th International Conference on Cloud Computing (CLOUD), 69–71. https://doi.org/10.1109/CLOUD60044.2023.00017
- Choochotkaew, S., Wang, C., Chen, H., Chiba, T., Amaral, M., Lee, E. K., & Eilam, T. (2023). Advancing Cloud Sustainability: A Versatile Framework for Container Power Model Training. 2023 31st International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), 1–4. https://doi.org/10.1109/MASCOTS59514.2023.10387542