Meta needs “ten times the computing power” for Llama 4.
- August 1, 2024
- 0
Meta is preparing to train the next generation of the Llama model, which the company expects will require ten times more computing power. Llama 3.1 has only just
Meta is preparing to train the next generation of the Llama model, which the company expects will require ten times more computing power. Llama 3.1 has only just
Meta is preparing to train the next generation of the Llama model, which the company expects will require ten times more computing power.
Llama 3.1 has only just launched, but Meta is already looking ahead to the next generation of Llama 4. Zuckerberg did so in the quarterly results the company released last night. Meta apparently has big plans for its next AI model, as Zuckerberg says it will require “10 times the computing infrastructure” to develop Llama 4. Llama 3.1 was trained on more than 16,000 Nvidia GPUs.
“It’s hard to predict how this will evolve over multiple generations. But at this point, I’d rather risk building capacity before it’s needed than too late, given the long lead times for launching new inference projects,” Zuckerberg said.
This will obviously cost Meta a lot of money. Meta spent $8.5 billion on capital investments last quarter, 33 percent more than last year. In one year, Meta spent almost forty billion dollars. It is difficult to determine what proportion of this spending can be directly attributed to the LLama 3.1 model.
Zuckerberg is already warning his investors that spending will reach another peak by 2025. Revenue is only expected after that. Unlike OpenAI, Google and Anthropic, Meta offers its models open source and therefore does not generate direct revenue from licenses. The company assumes that AI will open the advertising tap further in the long term.
Meta is the third AI powerhouse to announce its financials in the past week. We’ve seen the same trend with Google and Microsoft. Companies must first invest heavily in AI models and infrastructure before they can capitalize on it. Fortunately, Meta, Microsoft and Google can save a billion.
Source: IT Daily
As an experienced journalist and author, Mary has been reporting on the latest news and trends for over 5 years. With a passion for uncovering the stories behind the headlines, Mary has earned a reputation as a trusted voice in the world of journalism. Her writing style is insightful, engaging and thought-provoking, as she takes a deep dive into the most pressing issues of our time.