How Small Language Models Deliver Big Business Benefits

Forbes

09-05-2025

Small Language Models (SLM) are trained on focused datasets, making them very efficient at tasks like analyzing customer feedback, generating product descriptions, or handling specialized industry jargon.
There seems to be no limit to what artificial intelligence (AI) can help people do. But the tens of billions, even trillions of parameters used to train large language models (LLMs) can be overkill for many business scenarios.
Enter the small language model (SLM). SLMs are trained on relatively small amounts of specific data—fewer than 10 billion parameters or so. Because of their small size and fine-tuning, SLMs require less processing power and lower memory. This means they're faster, use less energy, can run on small devices, and may not require a public cloud connection.
Like LLMs, SLMs can understand natural language prompts and respond with natural language replies. They are built using streamlined versions of the artificial neural networks found in LLMs. But SLMs are trained on focused datasets, making them very efficient at tasks like analyzing customer feedback, generating product descriptions, or handling specialized industry jargon.
'LLMs are like a starship. It's very powerful and can go far, far away, but if you're doing something very tactical and specific, that starship is way too powerful,' says Neil Sahota, CEO of research firm ASCILabs and an AI advisor to the United Nations. 'If speed and costs are concerns, SLMs are the better way to go.'
The sweet spot for SLMs tends to be narrow tasks in high-volume niche applications or in low-power environments, such as on smartphones or Internet of Things (IoT) gadgets. They are also useful when data privacy is crucial, or internet access is sparse. For example:
Field service engineers don't always have high-bandwidth internet access. With an SLM on their device, they could use generative AI to query their field service manual. Low computational requirements and local processing make this possible.
Sales representatives might need to access a generative AI model containing sensitive data at a client site to provide tailored recommendations. An SLM could provide those results without the lag and potential privacy concerns that often come with using a mobile device.
Clinicians could use an SLM to analyze patient data, extract relevant information, and generate diagnoses and treatment options. The fact that data never leaves the device is a huge benefit for privacy.
But don't expect a significant shift from LLMs to SLMs. Organizations are more likely to implement a portfolio of models, each selected to suit a specific scenario.
AI developers, in fact, often work through a pipeline of models. A query might first go to an LLM, then to an SLM for classification, then back to the LLM to extract the information and generate a response.
At larger organizations, an LLM could be used for complex tasks—like developing a long-term business strategy that considers macroeconomic policies and global effects—while multiple SLMs handle dozens of business-unit-specific tasks such as analyzing consumer feedback and social media posts to guide new product development.
And while SLMs may be a cost-effective alternative to LLMs, they still have limitations. They don't understand complex language well, they lose accuracy when doing complex tasks, and they have a narrow scope of knowledge.
There are other trade-offs. While SLMs generally don't cost a lot to run, costs could add up if multiple SLMs are in use. 'If you have five models deployed and they're each using GPUs and occupying space and electricity in the data center, that costs more versus having one huge model,' says Sean Kask, AI chief strategy officer at SAP. 'Sure, the LLM uses a lot of electricity, but it's being used for a lot of different things, and you can refine data for smaller, more specific queries through prompt engineering.'
What's more, SLMs present many of the same challenges as LLMs when it comes to governance and security. 'You still need a risk and regulatory framework,' says Jim Rowan, head of AI at Deloitte Consulting LLP. 'You need an AI policy because you don't want business units using data and AI models without your knowledge. And you still have to set up guardrails because SLMs hallucinate too,' he adds.
SLMs also aren't necessarily easier to manage than LLMs. Even though the big AI players offer versions of SLMs through a service model where they provide the underlying engine, 'you still need people who know what the right data is. You need domain experts and a data scientist who can develop a good training strategy for the model,' Sahota says.
Companies will need to ask important questions before incorporating SLMs into their AI strategy:
What business case are you solving for? If the dataset is very small, controlled, and available, such as HR documents or product descriptions, it makes great sense to use an SLM. 'But if it's a large stack of constantly changing data or there's lots of variability in it, such as current mortgage rates or daily geopolitical events, you probably want to go the LLM route,' Sahota says.
What kind of performance and accuracy are needed? SLMs can be very accurate about straightforward questions, like an inquiry into current benefits. But if an employee says 'I would like to pay a third mortgage; can I draw off my 401(k)?' they may get a more generic answer. An LLM might be better at handling this type of question, as it could include information on HR and tax standards for 401(k) use.
What are your growth needs? Businesses need to anticipate how big the SLM might get over time. 'If you're a retailer and you're going to toss tens of thousands of products into the model over the next few years, that's certainly an LLM,' Sahota says.
As the number and type of available AI models continue to grow, businesses will need to understand the range of what's available to create their AI model portfolio.
'Choice is very important to your strategy,' Kask says. 'Pick the model that's right for you and for your embedded use case.'
This story also appears on SAP.com.