Explained: How is Intel challenging Nvidia's dominance with Gaudi 3

Intel unveiled the third generation of its artificial intelligence (AI) accelerator, Gaudi, at the Intel Vision 2024 event. The launch of Gaudi 3 occurred within a year of its predecessor, Gaudi 2. Demonstrating superior performance across various metrics, Intel asserts that its AI accelerator has surpassed Nvidia's H100, which was previously hailed as the 'most powerful' GPU for AI applications.

What is Gaudi 3?

Gaudi 3 is manufactured using a 5nm process and is designed for extensive AI computation. Each accelerator features a heterogeneous compute engine capable of executing 64,000 parallel operations, making them efficient for deep learning tasks. It supports various data types including FP8 and BF16.
Targeting large-scale enterprise applications, Intel states that Gaudi 2 will drive AI systems with tens of thousands of accelerators interconnected via Ethernet, with each Gaudi 3 accelerator integrating twenty-four 200-gigabit Ethernet ports. With its open software and standard Ethernet networking, Gaudi 3 facilitates scalability from single nodes to mega-clusters comprising thousands of nodes, supporting inference, fine-tuning, and training on a massive scale.

How does Gaudi 3 compete?

Intel's Gaudi 3 has been touted as a significant advancement over its predecessor, Gaudi 2. Promising a fourfold increase in AI computing power for BF16 and a 1.5x boost in memory bandwidth, Gaudi 3 is composed of two identical silicon dies interconnected by high-bandwidth links, each with 48MB of cache memory.

All said, Intel has highly marketed the release of a new AI accelerator on its superior performance over Nvidia’s GPUs.To be sure, Nvidia has managed to capture an estimated 80% of the AI chip market. 

In August 2023, then the executive vice president and general manager at the Data Centre and AI Group at Intel said that Gaudi 2 is competitive with Nvidia’s A100 processor but falls short on a raw performance basis in comparison with the relatively newer H100. “That said, Gaudi 3, which will be launched next year, would be able to directly compete with Nvidia H100 and further iterations,” said Rivera, who is now the CEO of Altera, an Intel company. 

Now, Intel says that Gaudi 3 delivers 50% on average better inference and 40% on average better power efficiency than Nvidia H100, at a fraction of the cost.

After launching the H100 in March 2023, this year Nvidia unveiled new chips such as the B100 and B200 GPUs. Notably, the GB200 superchip, a combination of two B200 GPUs, boasts a sevenfold performance increase over the H100.

When will Gaudi 3 be available?

By the second quarter of 2024 (Intel follows the January to December fiscal cycle), Gaudi 3 will be available to original equipment manufacturers (OEMs) such as Dell Technologies, Hewlett Packard Enterprise, Lenovo, and Supermicro. General availability of Intel Gaudi 3 accelerators is anticipated for the third quarter of 2024. Notably Indian companies such as Ola Krutrim, Infosys, and CtrlS are some of Intel’s partners for Gaudi accelerators.

In separate but related news, in March the US government agreed to provide $8.5 billion to chipmaker Intel through direct funding. Additionally, the company will also receive $11 billion in loans under the CHIPS and Science Act. 

The funding will support the construction and expansion of Intel’s facilities in Arizona, Ohio, New Mexico, and Oregon. It is also expected to create 30,000 jobs while creating ‘tens of thousands of indirect jobs. Commerce Department Secretary Gina Raimondo said that it is one of the largest investments ever in US semiconductor manufacturing. Raimondo also added that the administration hopes to US’ share of advanced chip production from 0% to 20% by 2030, as reported by Reuters. The announcement also includes funding for training and developing local workforce, including  $50 million in dedicated CHIPS funding.