After years of chatter, it’s time for Boomboom: Nvidia’s “superchip” Grace Hopper is finally rolling off the assembly line in bulk. The chip is an important part of Nvidia’s future strategy.

In spring 2021, Nvidia presented the Grace chip at the GPU Technology Conference: a home-built ARM processor tailored for the data center. It quickly became clear that Grace was not intended as an exclusive one-off. The main task of the CPU is to live on a “superchip” together with an Nvidia GPU. A fast connection between GPU and CPU as well as shared memory should make such a chip excellent for AI workloads. Nvidia has been talking about the Grace Hopper superchip with great enthusiasm for over a year.

specifications

Now the concept has also become a reality. Nvidia confirms that Grace Hopper is loudly rolling off the assembly line. The chip combines the ARM-based Grace CPU with the Hopper GPU architecture, which also supports the powerful H100 accelerator. The result is theoretically extremely effective System on a chip (SoC). Grace Hopper has the following specifications:

• 72 Arm Neoverse V2 CPU cores
• 528 GPU Tensor Cores (Hopper)
• 132 GPU streaming multiprocessors (hoppers)
• 900GB/s NVLink 4 CPU-GPU connection
• Up to 96GB GB HBM3 memory
• 4 TB/s GPU memory bandwidth
• Up to 480GB LPDDR5X ECC memory
• 512 GB/s CPU memory bandwidth
• TDP from 450 watts to 1,000 watts

Nvidia states that it uses the 4 nm process from TSMC when building the chip. The specifications above are for the current maximum configuration, which is only available in a limited edition. Most of the chips have to make do with a still strong 80 GB HBM3 memory.

The memory for the CPU does not come from classic DIMMs, but is contained on the superchip. This enables a higher clock rate and lower consumption, but of course you lose flexibility. The high TDP is striking, but it is difficult to compare it with the TDP of a classic CPU or GPU. After all, the superchip that Nvidia calls the GH200 covers almost the entire computer system.

great computers

The GH200 will also be available later this year. In anticipation, Nvidia is already announcing its own new DGX system. The DGX GH200 AI supercomputer is no problem and consists of 24 racks. About 256 GH200 chips fit in. Nvidia predicts an FP8 computing power of about one exaflop. The thing is a full-fledged supercomputer in the truest sense of the word. Note: This is not an exascale computer as HPC systems in the top 500 are measured using an FP64 benchmark.

The superchips in the DGX supercomputer will receive the full 96GB of HBM3 memory and will be the best units to roll off the TSMC production line. The GH200 typically ships with 80GB HBM3 as part of other servers. Nvidia appears to be disabling an HBM cluster on these devices, presumably to increase yield. The supercomputer will be available from the end of this year, although we doubt that there are many companies or institutions in our country that fall into the niche of the expected customers. “Classic” GH200 servers are built by other manufacturers such as Asus and Gigabyte.

Competition

With the superchip Grace Hopper, Nvidia is starting a new hardware race. AMD plans to release a similar chip soon, based on its own Zen 4 CPUs and the CDNA3 GPU architecture. This super chip will be called AMD Instinct MI300. TSMC is happy to see that, because the manufacturer will also bake these computer chips.

Intel, on the other hand, thought for a while that it wanted to start with a Falcon Shores XPU, but has already given up preventively. However, if Nvidia and AMD are correct, the superchips will become key components to support the explosive development of AI training and inference.