What are small language models and how do they differ from large ones?

Teresa Berndtsson / Letter Word Text Taxonomy / Licenced by CC-BY 4.0

By Lin Tian, University of Technology Sydney and Marian-Andrei Rizoiu, University of Technology Sydney

Microsoft recently released its latest small language model that can operate directly on the user’s computer. If you haven’t followed the AI industry closely, you might be asking: what exactly is a small language model (SLM)?

As AI becomes increasingly central to how we work, learn and solve problems, understanding the different types of AI models has never been more important. Large language models (LLMs) such as ChatGPT, Claude, Gemini and others are in widespread use. But small ones are increasingly important, too.

Let’s explore what makes SLMs and LLMs different – and how to choose the right one for your situation.

Firstly, what is a language model?

You can think of language models as incredibly sophisticated pattern-recognition systems that have learned from vast amounts of text.

They can understand questions, generate responses, translate languages, write content, and perform countless other language-related tasks.

The key difference between small and large models lies in their scope, capability and resource requirements.

Small language models are like specialised tools in a toolbox, each designed to do specific jobs extremely well. They typically contain millions to tens of millions of parameters (these are the model’s learned knowledge points).

Large language models, on the other hand, are like having an entire workshop at your disposal – versatile and capable of handling almost any challenge you throw at them, with billions or even trillions of parameters.

What can LLMs do?

Large language models represent the current pinnacle of AI language capabilities. These are the models making headlines for their ability to “write” poetry, debug complex code, engage in conversation, and even help with scientific research.

When you interact with advanced AI assistants such as ChatGPT, Gemini, Copilot or Claude, you’re experiencing the power of LLMs.

The primary strength of LLMs is their versatility. They can handle open-ended conversations, switching seamlessly from discussing marketing strategies to explaining scientific concepts to creative writing. This makes them invaluable for businesses that need AI to handle diverse, unpredictable tasks.

A consulting firm, for instance, might use an LLM to analyse market trends, generate comprehensive reports, translate technical documents, and assist with strategic planning – all with the same model.

LLMs excel at tasks requiring nuanced understanding and complex reasoning. They can interpret context and subtle implications, and generate responses that consider multiple factors simultaneously.

If you need AI to review legal contracts, synthesise information from multiple sources, or engage in creative problem-solving, you need the sophisticated capabilities of an LLM.

These models are also excellent at generalising. Train them on diverse data, and they can extrapolate knowledge to handle scenarios they’ve never explicitly encountered.

However, LLMs require significant computational power and usually run in the cloud, rather than on your own device or computer. In turn, this translates to high operational costs. If you’re processing thousands of requests daily, these costs can add up quickly.

When less is more: SLMs

In contrast to LLMs, small language models excel at specific tasks. They’re fast, efficient and affordable.

Take a library’s book recommendation system. An SLM can learn the library’s catalogue. It “understands” genres, authors and reading levels so it can make great recommendations. Because it’s so small, it doesn’t need expensive computers to run.

SLMs are easy to fine-tune. A language learning app can teach an SLM about common grammar mistakes. A medical clinic can train one to understand appointment scheduling. The model becomes an expert in exactly what you need.

SLMs are faster than LLMs, too – they can deliver answers in milliseconds, rather than seconds. This difference may seem small, but it’s noticeable in applications such as grammar checkers or translation apps, which can’t keep users waiting.

Costs are much smaller, too. Small language models are like LED bulbs – efficient and affordable. Large language models are like stadium lights – powerful but expensive.

Schools, non-profits and small businesses can use SLMs for specific tasks without breaking the bank. For example, Microsoft’s Phi-3 small language models are helping power an agricultural information platform in India to provide services to farmers even in remote places with limited internet.

SLMs are also great for constrained systems such as self-driving cars or satellites that have limited processing power, minimal energy budgets, and no reliable cloud connection. LLMs simply can’t run in these environments. But an SLM, with its smaller footprint, can fit onboard.

Both types of models have their place

What’s better – a minivan or a sports car? A downtown studio apartment or a large house in the suburbs? The answer, of course, is that it depends on your needs and your resources.

The landscape of AI models is rapidly evolving, and the line between small and large models is becoming increasingly nuanced. We’re seeing hybrid approaches where businesses use SLMs for routine tasks and escalate to LLMs for complex queries. This approach optimises both cost and performance.

The choice between small and large language models isn’t about which is objectively better – it’s about which better serves your specific needs.

SLMs offer efficiency, speed and cost-effectiveness for focused applications, making them ideal for businesses with specific use cases and resource constraints.

LLMs provide unmatched versatility and sophistication for complex, varied tasks, justifying their higher resource requirements when a highly capable AI is needed.

Lin Tian, Research Fellow, Data Science Institute, University of Technology Sydney and Marian-Andrei Rizoiu, Associate Professor in Behavioral Data Science, University of Technology Sydney

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The Conversation is an independent source of news and views, sourced from the academic and research community and delivered direct to the public.

The Conversation is an independent source of news and views, sourced from the academic and research community and delivered direct to the public.