In an article, Google DeepMind proposes a new method for training AI. This would be thirteen times more efficient and, above all, ten times more efficient than current technologies.

Google DeepMind proposes in a research paper JEST, short for Joint example selection. This is a new technique for training AI models. JEST is said to be thirteen times more efficient than current techniques for training AI and ten times more efficient. The latter is certainly very important, as AI training has caused the energy requirements of data centers, especially those of tech giants, to explode. Google itself has seen its CO2 emissions rise by 48 percent, rather than decrease, in five years.

quality before quantity

JEST is designed to help make AI training manageable. The model works in several steps. First, a small AI model is created that can divide the training data according to quality. The data is then grouped into batches. In a second phase, the large model begins training based on the batches of high-quality data. Preventive filtering according to quality reduces the need for quantity.

The availability of high-quality data is crucial for the JEST approach. In practice, this more complex technique is particularly relevant for larger players who want to train large models. This results in efficiency gains. Training GPT-4o already cost about $100 million. Anthropic’s CEO predicts that the cost of the next generation of models will be about $1 billion.