Meta releases lightweight versions of their Llama 3.2 models. These are designed for low power consumption devices.

Meta aims to popularize its open source large language models 3.2 1B and Llama 3B with these lightweight versions specifically designed for low-power devices. The models can be powered by energy efficient sources and still deliver strong performance.

Quantized models

Meta’s AI team emphasized that the models are designed for “short-context applications up to 8K” due to limited memory on mobile devices. Quantizing language models reduces their size by adjusting the precision of their model weights.

The developers used two different methods there, including “Quantization-Aware Training with LoRA adapters”. QLoRA helps optimize performance in low precision environments. If the model relies more on portability at the expense of performance, SpinQuant can be used. This optimizes compression to make model transfer to different devices easier.

Meta, in collaboration with Qualcomm and MediaTek, optimized the models for Arm-based system-on-chip hardware. Thanks to optimization with Kleidi AI kernels, the models can run on mobile CPUs. This enables more privacy-friendly AI applications where all processes run locally on the device.

The Llama 3.2 1B and Llama 3B quantized models are available to download today via Llama.com and Hugging Face. Meta also introduced video editing models earlier this week.