The hard-to-find Nvidia H100 GPUs already cost more than $40,000 today for those looking for one online.

The Nvidia Hopper H100 is currently the most powerful AI GPU, but the card is very hard to come by. The exorbitant prices on the used market testify to this. The PCIe version of the GPU is hard to find on eBay for less than $40,000. You could already buy the predecessor Nvidia A100 for less than 15,000 US dollars and even that was already included with a considerable margin.

In other words, the GPU is almost priceless. This is by no means due to the chip’s high performance, but rather to its limited availability. Nvidia isn’t getting the H100 GPUs off the assembly line fast enough. With the hype surrounding AI, everyone wants to train their generative AI models, and that requires GPUs.

Tens of thousands GPUs

Microsoft previously announced that OpenAI trains its GPT models on an HPC cluster in Azure with tens of thousands of GPUs. It recently became apparent that Elon Musk also wants to get back into the AI ​​world. For this he founded a new company and bought the necessary hardware: another 10,000 GPUs.

The scale at which the hardware required to conduct AI research is unprecedented. Nvidia can do its best, but it’s just not possible to put enough hardware on the shelves for everyone at a competitive price right now. That explains why a small PCIe card costs more than a car these days. Fortunately, the problems are currently limited to the H100 card. Classic GPUs are usually still available.