Recently, smartphone manufacturers have done that artificial intelligence features It requires huge investments to use it. Meta is also building a more compact language model for devices with limited capacity, such as mobile devices and tablets. With this model, the dimensions of efficient artificial intelligence are changing.

Meta Reality Labs team, PyTorch and Meta AI Research (FAIR) A joint team of collaborators was formed to develop a model with less than 1 billion parameters. This model is one thousandth of the larger models such as GPT-4. GPT-4 has more than 1 trillion parameters.

Artificial intelligence is getting smaller!

A person who shared a post about the topic via X Meta’s Chief Artificial Intelligence scientist Yann LeCun shared details of the research. The language model, called MobileLLM, will bring artificial intelligence to smartphones and other devices.

According to the research, smaller models are more suitable because of size than width. bring out the depth improves model performance. He also stated that they could increase the efficiency of storage-constrained devices by using advanced weight sharing techniques. In this process, it attracted attention with embedded sharing, grouped queue queries, and blocky weight sharing.

Another point that draws attention in the research is that only 350 million parameters It performed similarly to MobileLLM’s LLaMA-2 model with 7 billion parameters.