Intel has unveiled its Gaudi 3 AI accelerator at its Vision 2024 conference in Santa Clara, California.

The company also provided updates on its latest Xeon 6 processors, code-named Sierra Forest and Granite Rapids-D and first previewed at MWC earlier this year, and previewed its portfolio of new AI-optimized Ethernet solutions.

The Gaudi 3 product line is comprised of three hardware components. Firstly, the open accelerator module (OAM)-compliant accelerator card provides 1835 teraflops of FP8 compute performance, 128GB of HBM2e, 3.7TB/s of HBM bandwidth, eight matrix multiplication engines, and 1.2 TB/s of bi-directional networking.

The Universal Baseboard provides 14.6 petaflops of FP8 compute performance, more than 1TB of HBM2e, 29.6 TB/s of HBM bandwidth, 64 matrix multiplication engines, and 9.6TB/s of bi-directional networking.

Finally, the PCIe CEM is a new offering exclusively for the Gaudi 3 and contains 1835 teraflops of FP8 compute performance, 128GB of HBM2e, 600W of TDP, and eight matrix multiplication engines.

Connected through the common standard of Ethernet, Intel said the 5nm Gaudi 3 accelerator promises 2x more AI FP8 compute power, 4x more AI compute for BF16, 2x network bandwidth, and a 1.5x increase in memory bandwidth over its predecessor, the Gaudi 2.

Intel said the Gaudi 3 is already providing the foundation for the company’s next-generation processor, code-named Falcon Shores; a GPU for AI and high-performance computing (HPC).

According to the company, when compared to the Nvidia H100, it delivers 50 percent better inference on average across Meta's Llama2 models with 7B and 13B parameters, and OpenAI's GPT-3 175B parameter model. It also claims 40 percent on average better power efficiency across Llama 7B and 70B parameters, and the Falcon 180B parameter models.

The company also said the Gaudi 3 will also be a “fraction of the cost” of the Nvidia H100, but has not provided any pricing details.

Sample availability of the air-cooled Gaudi 3 and the liquid-cooled Gaudi 3 will be available in Q1 and Q2 of 2024, respectively.

Intel is expecting volume production availability of the air-cooled Gaudi 3 in Q3 of 2024 and is targeting Q4 of 2024 for the liquid-cooled option. The new PCIe offering will also be available in Q4 of 2024.

The Gaudi 3 accelerator will be available to OEMs, including HPE, Lenovo, and Supermicro, in the second quarter of 2024.

Intel provides updates to its wider AI product portfolio

Elsewhere, Intel announced that its Sierra Forest processors will be able to deliver a 2.7x performance per rack improvement and 2.4x performance per watt improvement when compared to second generation Intel Xeon processors. Scheduled to be released in the first half of this year, Sierra Forest is built on the Intel 3 process based on the Intel 7 node and offers up to 288 efficient-cores (E-cores) per socket.

Designed for Edge solutions, Granite Rapids-D incorporates software support for the MXFP4 data format, which reduces next token latency by up to 6.5x versus 4th Gen Xeon using FP16, with the ability to run 70 billion parameter Llama-2 models. It has built-in AI acceleration and will feature the latest generation of performance-cores (P-cores).

Intel also announced that it would be working with the Ultra Ethernet Consortium to lead on open Ethernet networking for AI fabrics. The AI-optimized Ethernet solutions being introduced include Intel AI NIC (network interface card), AI connectivity chiplets for integration into XPUs, Gaudi-based systems, and a range of soft and hard reference AI interconnect designs for Intel Foundry.

Finally, Intel unveiled new Edge silicon products across its Intel Core, Intel Core Ultra, and Intel Atom processors, in addition to a new Intel Arc GPU for the Edge. All of these products target the retail, industrial manufacturing, and healthcare sectors and will be available this quarter, supported by the Intel Edge Platform which is due to come to market later this year.

“Innovation is advancing at an unprecedented pace, all enabled by silicon – and every company is quickly becoming an AI company,” said Intel CEO Pat Gelsinger. “Intel is bringing AI everywhere across the enterprise from the PC to the data center to the Edge. Our latest Gaudi, Xeon, and Core Ultra platforms are delivering a cohesive set of flexible solutions tailored to meet the changing needs of our customers and capitalize on the immense opportunities ahead.”