AMD is expanding its market segments with a new AI model “AMD-135M” aimed at private, business use.

AMD announces its first small Lama-based language model AMD-135M, with which the company aims to expand its market segments. This model is aimed at private, business implementations and works based on speculative decoding. This is a technique in which a smaller “conceptual model” generates multiple candidate tokens in a single forward pass. The model exists in two versions: AMD-Llama-135M and AMD-Llama135M-Code. The company wants to open up new market segments in which its competitor Nvidia is not yet present.

Speculative decoding

AMD presents its first small AI model in a blog post: AMD-135M. According to AMD, this is the first small language model for the Llama family, trained from scratch on AMD Instinct MI250 accelerators with 670 billion tokens and divided into two models: AMD-Llama-135M and AMD-Llama-135M Code . The model focuses primarily on private, business implementations.

Additionally, the AMD Llama 135M code has been refined with 20 billion additional tokens specifically for encryption. This task was completed in four days. AMD’s models are fast because they use “speculative decoding”. The basic principle here is to use a small design model to generate a set of candidate tokens in a single forward pass. These tokens are then passed to a “target model” that verifies or corrects them. This allows multiple tokens to be generated simultaneously without sacrificing performance.

AMD believes that further optimizations can lead to even better performance. The company offers an open source reference implementation to stimulate innovation within the AI ​​community.