According to Stabilize AI, the Gaudi 2 accelerator is more efficient on training models than the Nvidia H100. Nvidia enjoys an advantage in inference.

Intel wants to compete with Nvidia in the lucrative world of accelerators. Stability AI now uses its own benchmarks to determine that the company’s Gaudi 2 accelerator is very competitive compared to the Nvidia H100.

Stability AI is the company behind Stable Diffusion. It tested both the training and inference capabilities of various chips in benchmarks based on its latest Stable Diffusion 3 model. This converts text into images, analogous to Dall-E.

Training

For training workloads, Stability AI provides a test setup with two nodes and sixteen accelerators each. Configurations with Intel Gaudi 2, Nvidia H100 and Nvidia A100 must handle the same workload. The result is undeniably in Intel’s favor. The Gaudi 2 accelerator is 56 percent faster than the Nvidia H100 and even 2.4 times faster than the older Nvidia A100.

Intel owes this victory to a series of design decisions that allow the Gaudi accelerator to swallow larger batch sizes than the other chips. But even with the same batch size, Gaudi 2 is the speed champion.

Inference

But the story is more nuanced. Stability AI also performed inference testing. A previously trained model must produce a result based on a prompt. This is where the optimization of Nvidia’s Tensor RT cores pays off. With these optimizations, the Nvidia H100 is about 40 percent faster than Gaudi 2.

Stability AI is not affiliated with Intel or Nvidia and has extensive expertise in generative AI with Stable Diffusion. The benchmarks therefore provide relevant insight into the performance of the chips. Nvidia has contributed a lot of knowledge and optimizations in recent years and is benefiting from it, but Intel is also doing quite well. The Gaudi 2 accelerator is competitive in inference and even superior in pure training workloads.