American startup Cerebras has trained “the largest AI model” on a single device equipped with a plate-sized Wafer Scale Engine 2 (WSE-2) chip. It says Tom’s Hardware.

“Using the Cerebras (CSoft) software platform, our customers can easily train modern GPT language models (such as GPT-3 and GPT-J) with up to 20 billion parameters in a single CS-2 system,” the company said. Declaration.

According to the initiative, Cerebras Weight Streaming technology allocates computing resources, allowing memory to scale to the size needed to store a rapidly increasing number of parameters in AI workloads.

“Models running on a single CS-2 are set up in minutes, and users can quickly switch between them with just a few keystrokes,” the statement said.

According to the company, storing up to 20 natural language processing models with billions of parameters on a single chip significantly reduces training and scaling overhead with thousands of GPUs. They added that this is one of the most painful aspects of NLP workloads and takes months to complete.

The Wafer Scale Engine 2 chip is built on a 7nm process, contains 850,000 cores, has 40 GB of internal memory with a bandwidth of 20 PB / s and consumes about 15 kW.

Recall that in April 2021 Cerebras introduced the WSE-2 processor designed for computation in the field of machine learning and artificial intelligence.

In August, the company created the CS-2 supercomputer. A setup based on WSE-2 chips can train an AI model with 120 billion parameters.

In May 2022, the Top500 ranking of the world’s most powerful supercomputers was managed by the American Frontier system developed by Oak Ridge National Laboratory. This is the first rig to hit a peak of 1.1 exaflops in the Linmark test.