A study conducted by Cast AI has found that companies only use around 13 percent of the CPUs they provision, and 20 percent of memory, on average.
The analysis - from a company that provides a Kubernetes cost optimization platform - is based on 4,000 clusters running on Amazon Web Services (AWS), Google Cloud Platform, and Microsoft Azure between January 1 and December 31, 2023.
The clusters included in the report are those of 50 CPUs or more. Little difference between cloud providers was found - with AWS and Azure both averaging utilization rates of 11 percent, though Google was slightly better, with 17 percent.
Memory utilization split between the big three cloud providers was 18 percent for Google, 20 percent for AWS, and 22 percent for Azure.
The utilization rate was slightly improved for larger clusters - with those containing 1,000 or more CPUs averaging 17 percent utilization.
Cast AI has suggested that the main reasons behind this are customers allocating more computing resources than necessary, customers reluctant to use "Spot Instances" due to perceived instability, and an under usage of custom instance sizes where you can select the CPU and memory ratio.
“This year’s report makes it clear that companies running applications on Kubernetes are still in the early stages of their optimization journeys, and they’re grappling with the complexity of manually managing cloud-native infrastructure,” said Laurent Gil, co-founder and chief product officer, CAST AI.
“The gap between provisioned and requested CPUs widened between 2022 and 2023 from 37 to 43 percent, so the problem is only going to worsen as more companies adopt Kubernetes.”
For cloud providers' bottom line, this reduces the need for power and to run the compute, while still seeing them earn revenue based on the hypothetical usage.
It does, however, drive up the need to invest in more compute and memory than is actually necessary, with every element of IT hardware having a carbon footprint through production and deployment. Additionally, by enterprises over-provisioning, others cannot access that compute for the duration.
It is not only the enterprises that over-provision.
Earlier this year, research from TechInsights found that in 2023 878,000 accelerators were responsible for turning out seven million GPU-hours of work, for an estimated $5.8 billion in revenue spending.
According to TechInsights, if the GPU clusters were operating close to capacity, the revenue figure would be much higher.
For example, AWS' UltraScale clusters are each comprised of 20,000 Nvidia H100 GPUs which can be rented for $98.32 per hour for a set of eight. Complete utilization would bring in revenue for Amazon alone of around $6.5bn.
Cast AI provides a Kubernetes automation platform for major cloud organizations, using AI to analyze and automatically optimize clusters in real-time.
