In recent months, Amazon, Google, Meta, and Microsoft have announced plans to ensure that their AI efforts enjoy reliable access to sources of energy. This AI energy imperative has risen as far as the Oval Office where both the Biden and Trump Administrations have both prioritized and made commitments to fuel AI infrastructure that requires access to massive amounts of energy.

The good news is that an AI energy “hack” already exists. And the solution is not in the cloud. It’s on-premises, in our own data centers.

It’s all about the data

It’s simple, really. The most efficient way to run any computational model, to run any workload, is to bring it to where the data is. Most enterprises do that already. They are running their workloads on-premises, in their own data centers. This is where their most sensitive proprietary data lives. The best and most efficient way to mine and utilize that data is to run those workloads where it is.

Early AI models were hosted in the cloud. For enterprises to play, they had to move their data to the cloud to get closer to those models. But enterprises should be doing it the other way around: they should bring their AI models on-premises to utilize their data in the most secure, energy-efficient, and cost-effective way.

Even if organizations prefer to take advantage of the elasticity of cloud computing for model training or fine-tuning, running those models on-premises adjacent to their data is more efficient, which can result in significant cost and energy savings.

Virtualization and energy efficiency

Instead of moving data and having that data be distributed or unnecessarily moved to different compute resources or taking it to one of the hyperscale cloud services, it is far more efficient to run that AI workload where the data is already created, processed, or consumed. This is what is known as private AI, and it offers a number of advantages for enterprise IT, including energy efficiencies.

Private AI allows enterprises to take full advantage of the virtualization stack that anchors their current data center infrastructure. Virtualization has helped enterprises achieve energy efficiencies for decades. Virtualization reduces the hardware footprint by consolidating workloads onto fewer servers and, by extension, consumes less power. This has long been one of the key advantages of virtualization and why most enterprise workloads today run on virtualization stacks. Organizations have reported an average of 49 percent infrastructure cost reduction, due partially to reduced hardware and software required to run IT services.

Virtualization brings these same energy efficiency advantages to AI workloads, where one can argue they are even more valuable. AI workloads are dependent on compute accelerators, like GPUs. They can be optimized, and like CPUs and memory resources, virtualization allows GPUs to be optimized and consolidated onto fewer physical servers and a smaller overall hardware footprint.

The result consumes less power while offering the same AI computing capabilities. Intelligently pooling and sharing GPUs and associated AI infrastructure (e.g. networking, storage, and memory resources) reduces the total hardware infrastructure required to run a variety of AI/ML services, which can, in turn, improve energy efficiency. Best of all, these benefits can be achieved without sacrificing performance.

Centralized organizational control

Enterprises that embrace a private AI approach have the additional advantage of having a centralized IT team manage and distribute their AI capabilities to the rest of their organization. This benefits the enterprise by providing organizational control over all its AI resources. This centralized control ensures that individual business units are prevented from standing up disparate AI clusters or servers within their own organizations. In addition, organizations can reduce risk by ensuring that only models that have been scanned and approved by central IT are placed into production.

The benefits of Private AI

Having the ability to consolidate AI and manage AI resources the same way an enterprise has always done makes it easier to deploy the AI model at a speed that best suits the organization. GPUs are a core technology that is needed for AI workloads, and most data centers are not yet equipped to run a very large number. Private AI ensures that the enterprise utilizes it as efficiently as possible.

Virtualization also allows enterprises to allocate their AI resources more smartly, securely, and more energy efficiently than purely hardware environments by enabling intelligent pooling and sharing of infrastructure capacity. Large enterprises today have literally thousands of virtual machines which are running at or below 30 percent of their utilization. When you get beyond Generative AI, many AI/ML services can run effectively on available CPU capacity, which can further reduce costs and energy consumption

For enterprises considering their AI deployment, a software-defined, virtualized private AI approach offers advantages that can deliver energy efficiencies, a reduced hardware footprint, and centralized operational control.

Subscribe to The Cloud & Hybrid Channel for regular news round-ups, market reports, and more.