Intel Launches Gaudi 3 Optimized for Large-Scale Generative AI

Along with the Xeon 6, equipped with performance cores, Intel also launched the new Gaudi 3 AI accelerator. Optimized for large-scale generative AI, the hardware features 64 Tensor Processor Cores (TPCs) and eight Matrix Multiplication Engines (MMEs).

Intel’s next-generation AI accelerator also offers 128GB of HBMe2 memory for training and inference, as well as 24 x 200Gb Ethernet ports for scalable networking. It also has a peak performance (FP8) of 1,835 TFLOPS and a TDP of 600W.

Compared to NVIDIA H100, Intel guarantees about 1.09x more performance in LLamA 3 8B, delivering 1.8x more performance per dollar. If we consider LLaMA 2 70B inferences, where Gaudi 3 is 1.19x better, the ratio rises to about two times the advantage for Intel’s solution.

“Demand for AI is driving a massive transformation in the data center, and the industry is calling for choice in hardware, software and developer tools”, highlighted Justin Hotard, Intel executive vice president and general manager of the Data Center and Artificial Intelligence Group.

Intel guarantees rapid availability of Gaudi 3

Additionally, there is support for the PyTorch framework and advanced Hugging Face transformer and diffuser models. Recently, Intel announced a collaboration with IBM to deploy Gaudi 3 accelerators as a service on the IBM Cloud.

Compared directly to its main competitor, the NVIDIA H100, Intel believes it offers a few advantages. The first is availability: delivery times for the H100 are around 54 weeks, while Intel can deliver to companies in a much shorter time frame.

Dell is currently co-designing RAG-based solutions leveraging Gaudi 3 and Xeon 6. These solutions, built on the Open Platform Enterprise AI (OPEA) platform, integrate OPEA-based microservices into a scalable RAG system optimized for Xeon systems and Gaudi AI, designed to enable customers to easily integrate Kubernetes and Red Hat OpenShift applications.