Meta launches Code Llama. Mark Zuckerberg’s company now has its own AI code generator.

Yesterday Meta presented its new open source LLM model Code Llama, an AI tool for coding. As the name suggests, Code Llama is based on Llama 2.

Practical

This new model is therefore fully tuned to coding. It is freely available for research and commercial use.

Users can let Code Llama generate and discuss code via text prompts. According to Meta, the model has the potential to help developers work more efficiently and ease the initial fears of those still learning to code.

So Code Llama is a version of Llama 2, but specialized in coding. The new model was trained with datasets designed for it. For example, Code Llama can not only generate code, but also natural language over code. You can use both code prompts and natural language prompts to do this.

In addition, Code Llama can also edit or even debug code. The model supports most major programming languages ​​such as Python, C++, Java, PHP, Typescript (Javascript), C# or Bash.

Multiple versions

Meta introduces three formats of Code Llama:

• 7 billion parameters
• 13 billion parameters
• 34 billion parameters

Each version is trained with five hundred billion code tokens and with code data. The first two formats are also trained with FIM knowledge (Fill in the middle) so they can integrate new code into existing code.

Another difference is that each format has different latency and server requirements. One GPU is sufficient for the 7m version, for example. The 34m version gives better results and is better suited for encoding support, but the other two are faster and better suited for lower latency tasks.

In addition, Meta launches two variants of the model:

• Code Lama–Python: a language-specific version trained with one hundred billion Python parameters because this programming language is the most tested code generation and because according to Meta it plays an important role in the AI ​​community
• Code Llama – Instruct: A more finely tuned and aligned version of Code Llama, trained with natural language input and expected outputs to help the model better understand what users expect from their prompts

AI for everyone

With Code Llama, Meta hopes to support software developers in everything from research and open source projects to industry and NGOs. In its communications, the company also welcomes the further development of the Llama 2 and related products and systems.

Meta, like many tech companies, is increasingly going for the AI ​​card. The company also recently introduced SAM, an AI model that identifies objects in images and videos.