Learn extra at:
In short: Small language fashions are usually extra compact and environment friendly than LLMs, as they’re designed to run on native {hardware} or edge gadgets. Microsoft is now bringing yet one more SLM to Home windows 11, as customers apparently want just a few AI-powered hints to assist them discover particular OS settings and customise their PC expertise.
Microsoft just lately introduced Mu, a brand new small language mannequin designed to combine with the Home windows 11 UI expertise. Mu will work alongside Phi Silica – the language mannequin beforehand launched on Copilot+ PCs – to energy the AI agent function in Home windows Settings that the corporate is currently developing.
Like Phi Silica, Mu will run solely on the system’s neural processing unit, delivering responses at over 100 tokens per second, which ought to present a clean consumer expertise in Settings.
Microsoft describes Mu as an environment friendly 330-million-parameter mannequin optimized for small-scale deployments. It’s primarily based on a transformer encoder-decoder structure, the place the encoder converts consumer enter right into a fixed-length latent illustration, and the decoder generates output tokens from that illustration.
Microsoft designed Mu as a extremely optimized “sibling” to Phi Silica, utilizing a number of strategies to cut back the parameter rely and enhance effectivity. The SLM was skilled utilizing an unspecified variety of A100 GPUs on the Azure Machine Studying platform, progressing by way of a number of phases and making use of strategies that Redmond developed throughout work on the Phi fashions.
After fine-tuning the mannequin, the builders discovered that Mu delivers efficiency almost corresponding to Phi-3.5-mini, regardless of being simply one-tenth its dimension. Moreover, Microsoft collaborated with silicon companions Intel, AMD, and Qualcomm to make sure that Mu is totally optimized for operating on native NPU-powered machines.
What was the purpose of all these optimization efforts? In line with Redmond, Mu will energy the brand new “agentic” expertise within the upcoming AI-enhanced Settings app. The AI and cloud big goals to supply a brand new “chat” function that lets Home windows customers ask settings-related questions in pure language and obtain efficient solutions, even once they barely know what they’re on the lookout for.
Agentic AI within the Settings app is meant to enhance Home windows’ ease of use. Microsoft acknowledged that coaching an SLM able to dealing with (and modifying) a whole lot of system settings was a problem, and the corporate says it welcomes consumer suggestions on the brand new function.
Personally, I by no means skilled any usability or discoverability points with Settings throughout eight years of utilizing Home windows 10, but Microsoft stays bullish about reworking Home windows into an “AI-first” platform.
