With DBRX, Databricks joins the wide range of LLMs as a new player. The model is offered open source and, according to the company, is the most advanced in this category.

Databricks is fully engaged in the AI ​​debates and today launches its LLM DBRX. The company invested ten million euros in training the model, and the necessary financial resources were saved last year. DBRX consists of 132 billion parameters and is completely text-based: the model can only process and produce text.

In the competitive AI market, it’s bold to call yourself the best, but Databricks has found a category where it excels. According to the company, DBRX is the most advanced open source LLM available today. It shares benchmarks in which the model proves to be better in language skills, programming and mathematics than Meta’s LLama2 (which has already received a successor LLama 3), Elon Musk’s Grok-1 and the models of the French Mistral, which is also a friend too home. Databricks.

Mixture of experts

Databricks DBRX was designed according to a Mixture of experts-architecture. This means that the model is divided into several sub-models, each trained in a specific domain. When you ask DBRX a math problem, it calls the model that is good at math and passes questions about the software code to the model that was trained on it. This essentially makes DBRX easier to operate.

“Easy” is an overestimation, because running DBRX still requires at least four Nvidia H100 GPUs, which the average company doesn’t have in its drawer. Databricks will also offer the model through its own Mosaic AI platform. For Databricks customers, this may be the most accessible way to try out DBRX.

Open vs. closed

Databricks doesn’t dare compare the DBRX benchmarks with GPT-4 or Google Gemini, but believes it can compete with the two giants thanks to the model’s openness. The model is offered open source via GitHub and Hugging Face for research and commercial purposes.

“The most valuable data is in companies. This is a sector where high training requirements make the barrier to entry too high for small businesses. Right now, only the well-capitalized companies are willing to play. We’re trying to make AI possible for everyone with open source models,” CEO Ali Ghodsi told SiliconAngle.