AMD introduces the MI300X. It’s not an APU like the MI300A, but a thoroughbred GPU that’s overflowing with RAM. That gives the plentiful memory with the accelerator an advantage over the Nvidia competitor and the H100.

At first there was only the AMD Instinct MI300: an APU that combines Zen 4 cores with CDNA 3-based graphics chiplets and shared HBM 3 memory. At the beginning of this year, the chip was suddenly called AMD Instinct MI300A, although the concept changed little with the addition of “A”.

AMD Instinct MI300X

At the data center and AI technology premiere in San Francisco in the USA, we suddenly hear that the Instinct MI300 is one family and the MI300A has a brother: the AMD Instinct MI300X.

With the X version, AMD takes advantage of the accelerator’s flexible modular chiplet design by ripping the 24 Zen cores off the chip and putting additional CDNA 3 GPU cores in their place. In other words, the MI300X is not an APU, but a relatively classic accelerator that you need to connect to a processor in a server.

Complete model in one chip

The MI300X features 192GB of HBM 3 storage with a bandwidth of 5.2TB/s. “The chip is optimized for generative AI,” says CEO Lisa Su on stage, and during a demo we can see exactly what that means. The entire Falcon 40B model fits in the huge built-in memory at once. For example, the chip can also draw conclusions about very extensive AI models.

The MI300X has 2.4x more memory and 1.6x more HBM bandwidth than the Nvidia H100. We doubt whether the chip is also more powerful. The MI300X is a guest teaser at the San Francisco event. The actual launch will follow later, so AMD does not feel compelled to reveal specifications such as TDP or computing power in flops.

These specs are also somewhat irrelevant to AMD’s market positioning of the Instinct MI300X. “We have a TCO advantage,” said Brad McCredie, VP Datacenter and Accelerated Processing at AMD. “In terms of conclusions, we’re certainly going to have an advantage in terms of conclusions per dollar.”

Responding to changing workloads

McCredie admits splitting the Instinct MI300 into the MI300A and MI300X wasn’t planned from the start. “But it was always an option.” So why the development from a product to a product family? “AI workloads have evolved rapidly and become more computationally intensive. That’s why we chose the MI300X, which wasn’t all that difficult thanks to the modular design.”

For a competitive functional analogy, consider the Nvidia H100 as the MI300X counterpart and the Nvidia Grace Hopper superchip as the MI300A equivalent.

platform

Meanwhile, on the main stage, Su shows another platform design with eight MI300X chips in standard server design. In her opinion, this can be implemented quickly and easily in the existing infrastructure. Announced at CES, the Instinct MI300A is already being tested by customers today. The Instict MI300X allows partners to get started in the third quarter of this year. Both Instict MI300 variants are scheduled to go into series production by the end of the year.