Meta and Arm team up to deliver improved performance of LLM Llama 3.2 on Arm-based CPUs

The Artificial intelligence is booming, the devices make use of this technology to deliver better results and user experiences. To further improve this experience, Arm has collaborated with META to offer best results with Llama 3.2 when running on Arm-based processors. This is the result of the investment in AI that Arm has made together with open source softwarewhich allows creating new LLMs from 1B to 90B under Arm platforms.

He New Llama 3.2 3B can be run on Arm-based mobile phones with a processing Up to 5 times faster and generating 3 times more tokensthis gives 19.92 tokens per second. As a result, latency is reduced for AI tasks by offering faster results. It has also been achieved reduce consumption processing AI at the edge, saving energy on data transmitted to and from the cloud.

Being open source, Llama 3.2 can Get improvements and optimization from the community. Arm is working to optimize AI performance on these CPUsoffering compatibility with the Cortex and Neoverse cores without the need for developer integration. There are also benefits when training with these processors, with the integration of Kleidi (libraries for AI) with PyTorch and ExecuTorchgenerating lower times on these Arm-based processors such as AWS Graviton.

This collaboration allows Leverage available resources and improve AI tasks using Meta’s LLM on Arm CPUs. An upgrade available for home users or businesses, which will offer help and support to carry out day-to-day activities in a more productive thanks to the use of Artificial Intelligence.

End of Article. Tell us something in the Comments!