Recommendations for performing energy measurements

WIP

This list of recommendations is work in progress…

Use an appropriate tool for what you want to measure

See Measure energy consumption of software#Overview of Tools.

Observe idle consumption

It’s important that the energy consumption of the machine in idle mode (baseline) is the same between runs, so it doesn’t influence the results.
Therefore, always measure the consumption in idle mode before performing your actual measurement.

Use percentages instead of raw values

To compare results between measurements, use percentages instead of raw values.

noureddine.etal.2015.monitoringenergyhotspots (pg. 26)

These results outline the importance of using percentages when comparing energy consuming of software code. This is mainly due to the different hardware that machines use, thus consuming different amount of energy while still keeping similar energy trends and distribution in software.

noureddine.etal.2015.monitoringenergyhotspots (pg. 31)

The goal of our approach is to observe trends in energy consumption and profile applications to detect energy hotspots. Therefore, we argue that using percentages when comparing energy consumption of methods and classes is more useful and representative than raw values. Our approach is thus useful in profiling applications in order to find the origin of energy leaks. Developers can then provide hotfixes for the application in order to reduce its energy footprint.

Estimate network transmission costs

If you have a distributed system, you can't measure the transmission costs of the network communication. However, you can measure the amount of bytes that is transmitted between your services and to external services. With this value you are able to estimate the network transmission costs with a formula:

B y t e s [G B] \times C o n s t a n t F a c t o r [k W h / G B]

Constant factor depends on your system boundaries. Cost of network infrastructure transferred through the internet (eg. routers, cables, data-centers etc.) for 2023: 0.00375 kWH / GB

Source: https://www.green-coding.io/co2-formulas/#gigabytes-to-kwh

Warming up of applications

In performance benchmarking of Java applications, warming up of the runtime environment is crucial. Depending on your measurement goal and your application under test, you should consider warming up of the application before executing your energy measurement.

For example, if you want to compare the energy consumption of a Java application and an application compiled to machine code, you need to warm up the Java application to make a fair comparison under runtime.

Per-Process Measurements

Measuring how much energy an individual process has consumed is not straightforward and poses a number of challenges.
See Measure energy consumption of software per process for more information.

You should ask yourself what the goal of your energy measurement is and whether per-process measurement is necessary at all. For example, the Green Metrics Tool follows the philosophy that all components involved in the execution of a standard usage scenario should be measured to reflect actual use cases of the software, therefore it doesn't support per-process measurements.