Meta Develops Efficient Language Models For Smartphones

(RTTNews) - Meta (META) has created a new compact artificial intelligence model named MobileLLM for smartphones and other devices with limited computing power.

MobileLLM, developed by Meta Reality Labs, Meta AI Research (FAIR) and PyTorch, has less than one billion parameters.

Highlighting the key aspects of the study, Yann LeCun, Meta's Chief AI Scientist, posted on X/Twitter, "Our findings indicate that, for smaller models, prioritizing depth over width enhances model performance. Furthermore, by leveraging advanced weight-sharing techniques, including embedding sharing, grouped query attention, and block-wise weight sharing, we achieve significant enhancements in weight utilization within storage-constrained scenarios".

The design improvements have helped MobileLLM beat previous models of the same sizes by 2.7 to 4.3 percent, according to benchmark tests.

According to LeCun's post, MobileLLM exhibits "substantial advancements in zero-shot commonsense reasoning, question answering, and reading comprehension tasks compared to SoTA methods".

Moreover, the researchers demonstrated the effectiveness of MobileLLM models in chat and API calls, "underscoring their adeptness in handling such tasks".