Training GPT-4 cost $100 million and involved a massive dataset containing billions of words and text.

OpenAI has become the company that brought AI to the masses. Nvidia CEO Jensen Huang called ChatGPT “the iPhone moment” for AI. With the introduction of GPT-4, OpenAI presented its most advanced model to date last month. This project cost around $100 million to train the AI ​​model on tens of thousands of powerful GPUs.

Sam Altman, CEO of OpenAI, says during an MIT event that he doesn’t expect progress in the future by developing larger and larger models. “I think we’re nearing the end of an era that’s been about such gigantic, gigantic models. We will make them better in other ways.”

Altman’s words suggest that GPT-4 is the latest general evolution in OpenAI’s strategy to make models bigger and bigger with more data. In an article describing GPT-4, the company claims that model size scaling has limited benefits. OpenAI must also consider the physical limits of how many data centers the company can build and how quickly they can be built.

AI experts tell Wired that there are other ways to improve such models besides increasing the size of the data set. New models or architectures for developing an AI model offer many interesting possibilities. There are also ways to make the current models much better and more user-friendly without having to add additional parameters.

No GPT-5 in development yet

Earlier this week, Altman said GPT-5 is not currently being trained. Additionally, OpenAI doesn’t plan to start doing this any time soon either.

That doesn’t mean that research on generative AI has suddenly been put on hold. Finally, OpenAI is diligently working on the capabilities of GPT-4 and also pay attention to better security for the model. It’s unclear what else the company can achieve with this model. In theory, OpenAI can still introduce many improvements without adding a new version number behind its GPT model.

The predecessor GPT-3 also received an upgrade with GPT 3.5. It is quite possible that we will see even more improved variants of GPT-4. Also, GPT-4 is still very new. It is far from clear where the limits of this model lie. Even without GPT-5, OpenAI will continue to be at the forefront of AI research for a while.