The best-performing products, the best price, and the best underlying technology: AMD has good reason to prove why it’s a major player in the CPU and AI market. There’s no shortage of announcements in San Francisco either. Will the ambition pay off?

At the data center and AI technology premiere in San Francisco, USA, we almost drown in cores and HMB3 memory. CEO Lisa Su takes the spotlight as she personally reveals AMD’s ambitions for the future to the assembled international press. “Today is an important day for us as we take a major step toward making AMD the partner of choice for data centers and AI.” The company is hungry and sees no reason why it isn’t at the forefront of the industry with its CPUs and GPUs food chain should stand.

war on all fronts

That’s easier said than done. Where Intel CEO Pat Gelsinger is shutting down businesses left and right to refocus the company on its goals coreBusiness (pun), AMD in San Francisco decides on all-out war. Su and her companions compete for leadership with CPUs, GPUs, FPGAs, and DPUs. If it can do calculations and contains a microtransistor, AMD’s goal is to have the best variant on the market.

The definition of “best” is quite subjective, but at AMD we see a clear line. Although the manufacturer always boasts of sky-high performance with all new components, this is actually the case total ownership cost (TCO) for the customer in focus. AMD wants to put as much horsepower as possible in the hands of customers, at the best possible price and with the lowest possible power consumption.

Bergamo

For this purpose, the manufacturer has a not-so-secret weapon: chiplets. During her presentation, Su repeatedly referred to the importance of chiplet technology for the rapid expansion of AMD’s portfolio. The announcements at the premiere are almost all due to chiplets as a central design philosophy.

For example, AMD is showing its first data center GPUs specifically designed for cloud-native workloads: Epyc Bergamo. Bergamo uses a variant of the Zen 4 cores in the general 4th generation Epyc processors: Zen 4c. However, Zen architect Mike Clark insists that Zen 4c Bergamo is little different from Zen 4 Genoa.

The performance of the chiplet

“We’ve reduced the available cache per core and clock speed for Zen 4c, so the core itself is smaller,” he says. “As a result, twice as many cores fit on a CCD (core chiplet matrix).” For Bergamo, AMD takes a Genoa chip and replaces the Zen 4 CCDs with Zen 4c CCDs. Everything else including the I/O part stays the same, ensuring platform compatibility. And perhaps more importantly, research and development costs have remained relatively contained for AMD.

We don’t need a completely new architecture to create an economic core.

Mike Clark, Zen Architect

“We don’t need a completely new architecture to create an economic core,” laughs Clark. On the podium, Su can now come up with nice benchmark numbers, where Bergamo not only trumps the corresponding chips from Intel, but also Ampere’s cloud-native Altra CPU, which is based on ARM.

Incidentally, Dan McNamara, SVP of AMD’s Server Business Unit, makes a brief appearance on the main stage to introduce Genoa-X. This variant of Epyc has up to 1.1 GB of cache on board, but here too it is not fundamentally different from Epyc Genoa. The cache was replaced by the second generation of AMD’s 3D V-Cache, but the framework for the Epyc modular chip was designed for this from the start. Engineers could simply swap out the cache component and voila: a new chip for new workloads.

More than an MI300

Then it’s on to AI land. “AI is the big megatrend of this era,” suspects Su. The computing power demand for AI training and inference is certainly greater than AMD predicted not too long ago when it proposed the Instinct MI300. This chip would be an APU that combines the GPU and CPU into one.

Today the thing is called the MI300A and its accelerator performance is no longer sufficient for the toughest tasks. According to Brad McCredie, VP of Datacenter and Accelerated Processing, the solution wasn’t that complex. “It was pretty easy to make the MI300X. We always had that option in the back of our minds. We removed the Zen 4 cores from the Instinct MI300A and replaced them with additional CDNA 3 cores.”

Thanks to the chiplet approach, AMD had a classic AI accelerator ready in no time. Then the chip designer packed 192 GB of HBM 3 onto the chip. Su beams as she shows what that means during a demo. “The entire Falcon 40B AI model fits in the chip’s memory, so an Instinct MI300X is enough for the conclusion.”

Good enough for Frontier

Of course, powerful chips remain worthless if nobody accepts them. AMD is also aware of this. So Su and his colleagues are taking the time to bring the mature software framework home. AMD’s ROCm software is now at version five, and if it’s good enough for the exascale supercomputer Frontier, it might be good enough for you, it says between the lines.

AMD consistently relies on cooperation with the open source world for its AI software offerings and tries to differentiate itself from Nvidia. Clement Delangue, CEO of Hugging Face, is present to emphasize the importance of this fact. “Hugging Face is sort of the GitHub for open AI models,” he says. “We are now working with AMD to ensure that Hugging Face models also work seamlessly on AMD hardware.”

AI in the laptop

Thanks to the integration into the open source world, AMD’s accelerator hardware should become interesting for a wider audience. There is some catching up to do, because Nvidia dominates the market with a share of around 80 percent.

What Nvidia doesn’t have are laptop chips. AMD has one with Ryzen and Ryzen Pro and is striving for one Ryzen AI engine to stop. This is a chiplet that serves as a built-in NPU (Neural Processing Unit) and can speed up the AI ​​on the laptop. The first implementations are currently being worked on.

It is no coincidence that the architecture of the NPU is similar to that of the Instinct MI300 series. Chiplets and unified architecture are likely to seduce the market. “The engine will allow us to innovate faster,” said David McAfee of Ryzen Product Group. “We use the same AI architecture from endpoint to cloud.”

Programmable DPU

Then on to the DPU. AMD has less to say about that in San Francisco, even though AMD Pensando is in the spotlight. Pensando is an acquisition by AMD, with which the chip designer bought into the upper end of the DPU market in one fell swoop.

Numerous large guests cross the stage to sing a hymn to Pensando. The programmability of the DPUs enables new possibilities of protection and is alone responsible for savings in the millions, it is said.

AMD itself maintains a predictable clock speed. The company is consistently bringing its innovations to market according to the proposed roadmap. The advanced chiplet design plays an important role here.

Consistently top components

The results are impressive. On the CPU side, there’s Epyc. Epyc Genoa is the most powerful processor for general workloads, Genoa-X supports technical workloads and Bergamo uses the cloud natively.

For the AI, we then have the Instinct MI300A APU and the MI300X accelerator, paired with an open software ecosystem. The Ryzen AI engine brings AI architecture from the cloud to the endpoint: a unit that no other vendor can match today.

AMD has not lost sight of the network either. AMD Pensando combines DPU and ASIC into a single unit that can offload security and virtualization workloads from server CPUs. With the Giglio architecture, the successor is already around the corner, but Giglio DPUs will not have any compatibility issues with existing Pensando DPUs.

Confidence in the benchmarks

AMD’s top generals release benchmark slide after benchmark slide to demonstrate the superiority of all their hardware over Intel and Nvidia equivalents, but it’s not high performance that should win the market over time. Along with chiplets, another word keeps popping up: TCO. All of AMD’s components tend to offer the best performance for the money and generally per watt. “We have a TCO advantage,” McCredie points out. “And certainly in terms of the conclusion per dollar.”

We have a TCO advantage, especially in terms of inferences per dollar.

Brad McCredie, VP Data Center and Accelerated Processing

In San Francisco we feel the result. AMD is brimming with confidence. All products are ready to serve customers across the spectrum, and those who sign up get high performance at a lower total cost of ownership over the lifetime of a component. The market share is not surprising, although AMD still eats significantly less cake than Intel or Nvidia.

Are customers following?

AMD may already deserve to be crowned processor king and AI emperor, but it still has a long way to go. At this point, CEO Lisa Su shares a powerful story. More and more servers and instances with AMD inside are appearing on the battlefield and the advance will not stop immediately. Still, the premiere of data center and AI technology in San Francisco is an outsider event, unwarranted as that may be.