Server Utilization

Definition

The amount of work that is being performed relative to the total capacity of the IT equipment or data center capacity, expressed as a percentage. Utilization can also be applied to power and cooling system capacity. Tracking of average IT equipment utilization is a good sustainability metric, as the work delivered per unit of energy increases with an increase in IT equipment utilization.

— Jay Dietrich. (2022). Glossary of Digital Infrastructure Sustainability. Uptime Institute.

Average server utilization

Deng et al. (2018)

@Deng.etal.2018.ResourceUtilizationAnalysis

Utilization of servers in the Alibaba cloud in 2017:

• CPU:
  • Average:
    • 90% of nodes average usage did not exceed 35%
    • 90% of nodes’ 95th percentile usage was less than 50%
  • Minimum:
    • minimum usage of all nodes had less than 30%
    • minimum usage of 90% of nodes did not exceed 15%
• Memory:
  • Average: 90% of nodes’ average utilization rate did not exceed 60%
  • Minimum: 90% of nodes’ minimum usage did not exceed 45%.
  • 95% of nodes’ 95th percentile memory usage was less than 70%.

Shehabi et al. (2016)

@Shehabi.etal.2016.UnitedStatesData

Actual server utilization levels of the big cloud providers are unknown. The tool Cloud Carbon Footprint uses the average 50 % as a fallback if the cloud provider's API don't provide actual utilization data.

NRDC 2014

2014 Data Center Efficiency Assessment from the NRDC

Even though the past decade has seen data center IT managers adopt server virtualization broadly (a technique to consolidate underutilized servers), average server utilization is still between 12 and 18 percent and has remained static from 2006 through 2012.

Although hyper-scale cloud providers can realize higher utilization rates (ranging from 40 to 70 percent), even they are not consistently achieving those rates. [23] New research from Google indicates that typical server clusters average anywhere from 10 to 50 percent utilization. [24]

[23] Whitney et al., The Carbon Emissions of Server Computing for Small- to Medium-Sized Organizations: A Performance Study of On-Premise vs. the Cloud, NRDC and WSP Environment & Energy, October 2012.

[24] L.A. Barroso and Urs Hölzle, The Data Center as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2nd ed., Morgan & Claypool, 2013.
→Third edition: @Barroso.etal.2019.DatacenterComputer

More secondary sources:

The 2014 Data Center Efficiency Assessment from the NRDC has cloud server utilization at 65% and on-premises utilization running 12 to 18%, which is consistent with other estimates I’ve come across over the years.
— Jeff Barr (2014) Cloud Computing, Server Utilization, & the Environment | AWS News Blog

Server utilization rates of the hypercloud providers are commonly over 65 % versus between 10% to 20% average on prem.
— Operational Choices in Sustainable Architecture by the Green Software Foundation

Container Workloads

Source: https://www.datadoghq.com/container-report/

Mainframes

In fact, mainframes generally have the highest resource utilization rates across all hardware: mainframes frequently exceed 95% of their resource utilization. This means that practically everything is in use, which means you need fewer additional hardware products, which reduces your carbon footprint and energy thresholds. (By contrast, other IT systems are designed to run at no more than 70% capacity or less, saving the rest of their resources for self-generated tasks like clean up and maintenance.)
— Why Mainframes are Key to Green IT & Environmentally Sustainable Computing – BMC Software | Blogs