The world of generative AI (GenAI) is evolving rapidly and there are more and more models that companies can use. These models can be divided into two main categories, namely closed-source models and open-source models.

Closed-source based models such as OpenAI’s GPT-4o, Anthropic’s Claude 3 or Google’s Gemini 1.5 Pro are developed and operated by private and public companies. These are known for their high performance and training on large data sets, but also have limitations in terms of customization, management and cost.

On the other hand, open source-based models such as Llama 3 or Mistral are freely available for business use and customization. These models offer greater flexibility, transparency and cost efficiency compared to their closed source counterparts.

Advantages and challenges of closed source models

Closed-source models have become very popular due to their wide range of applications and ease of use. Platforms such as OpenAI’s API or Google Cloud AI allow companies to access powerful GenAI models without the need for extensive in-house expertise. These models are ideal for a wide range of tasks, from generating different types of content to automatic translations.

However, using closed source also brings challenges. Companies have limited control over the architecture, training data, and output of the model. This lack of transparency can raise concerns about privacy, security, and bias. In addition, the cost of these models can rise quickly as usage increases, making it difficult for companies to quickly scale their GenAI applications.

Rise of Open Source: Customization, Control and Cost Efficiency

Open source-based AI models are an attractive alternative to the more closed models and are increasing their use. According to GitHub, between 2022 and 2023 there was a 148% increase in unique contributors and a 248% increase in the total number of open source GenAI projects on GitHub. With open source AI, companies can adapt the models to their specific needs. By training them on their own data, you can also create custom GenAI applications that are significantly more powerful than generic closed source models.

Open source based AI models are an attractive alternative to the more closed models.

Rein de Jong, Regional Vice President BeNelux at Cloudera.

In addition, AI applications based on open source provide companies with full control over the implementation and use of the model. According to data collected by Andreessen Horowitz (a16z), 60% of AI leaders cited control as the main reason for using open source. With this control, companies can ensure data privacy, security, and regulatory compliance. In addition, open source-based models also enable significant cost savings compared to closed source models, as these models can run and scale on native infrastructure without excessive operational costs.

Choosing the right GenAI model depends on several factors, including the application itself, available data, performance requirements, and budget. For some applications, closed-source models may be the best fit due to their ease of use and performance. For organizations that require greater customization, control, and cost management, open-source models are typically preferred.

Cloudera’s approach to model flexibility and deployment

Cloudera recognizes the importance of flexibility in selecting and deploying GenAI models. Our platform supports a wide range of open source and closed source models, so organizations can always choose the best model for their specific needs.

Cloudera also enables organizations to easily train, refine, and deploy open source AI models on their own infrastructure. The platform provides a secure and managed environment for model development and enables data scientists and engineers to collaborate effectively. Cloudera’s platform also integrates with open source libraries and frameworks such as TensorFlow and PyTorch, ensuring compatibility with the latest developments in GenAI.

For enterprises that prefer closed-source models, Cloudera’s platform offers seamless integration with the most widely used public cloud AI services, such as Amazon Bedrock. This integration enables enterprises to leverage the power of closed-source based models while maintaining control over their business data and infrastructure.

This is a contribution by Rein de Jong, Regional Vice President BeNelux at Cloudera. The publishers are not responsible for the content.