What is a small language model (SLM)?

A model in the roughly 1B to 15B parameter class, built for depth and repetition on a narrow task rather than broad reasoning. It is fine-tunable, cheap to serve, and small enough to run on-prem or on modest hardware. Treat it as a distinct deployment class, not a shrunken general-purpose model.

Are SLMs worse than LLMs?

Only on broad, open-ended reasoning, where a frontier model is stronger. On a narrow task it is fine-tuned for, a small model is frequently faster, far cheaper and often more accurate than a generalist giant. It trades breadth for low latency, on-prem privacy and low cost. Worse and better are the wrong frame: it is a different tool for a different job.

When should I run a model on-device?

When privacy is non-negotiable or the workload must run offline. An on-device model keeps data on the laptop or phone, removes per-call cost entirely, and needs no network. The tradeoff is capability, so it suits narrow, well-defined tasks: classification, local assistants, on-device extraction. For broad reasoning, stay on a cloud tier.

Reasoning model or instruct model?

Instruct models execute a request directly: reasoning models spend extra inference-time compute to plan first, trading higher cost and latency for stronger multi-step reasoning. Reasoning models plan, instruct models do. This is a model type, orthogonal to tier: you can have an instruct or a reasoning variant within the same capability rung.

What is Mixture-of-Experts (MoE)?

An architecture where a router activates only a sparse subset of the model parameters per token, rather than all of them as a dense model does. The result is the capacity of a large model at the compute cost of a much smaller one. Like reasoning vs instruct, it is a within-tier lever: it shifts the cost and latency point, not the tier itself.

← Stack

Model tiers

Frontier, workhorse, small, edge: a capability ladder, and matching the rung to the task is the lever.

Overview

Models are not one market, they are a ladder. At the top sit frontier models: the most capable, the best at the hardest reasoning, and the most expensive and slowest per call. Below them is the workhorse tier, the mid-sized models that do most production work at a fraction of frontier cost: this is where everyday tasks belong. Lower still are small language models, the roughly 1B to 15B class you can fine-tune and run on-prem or on modest hardware. At the bottom is the edge tier: a quantized model running on a laptop or a phone, fully offline, with zero per-call cost. Each rung trades raw capability for lower cost, lower latency, and the option to keep data closer to you.

The common mistake is to read this ladder as quality descending into junk, with small models a consolation prize for teams that cannot afford the good stuff. That is wrong. A small language model is a different deployment class, not a failed large one: it trades general breadth for low latency, on-device or on-prem privacy, and dramatically lower cost on high-volume, predictable workloads. It wins where data must never leave the perimeter, the case in regulated healthcare, finance and legal work, and it wins on narrow repetitive tasks where a fine-tuned small model is faster and often more accurate than a generalist giant. Built for depth and repetition, not breadth.

The lever is this: tier is chosen per task, not per company. A single product can call a frontier model for the one hard reasoning step, a workhorse for everyday generation, and a small model for high-volume classification, all in the same flow. The cost gap between frontier and small runs roughly 10 to 20 times, so the saving from sending each task to the cheapest rung that can do it is large. Deciding which rung per request is exactly what model routing exploits, and that mechanism has its own page: choosing-a-model goes into the routing detail. Here we map the ladder itself.

Architecture

Read the ladder top to bottom. As you descend from frontier to edge, capability narrows from broad reasoning to a focused task, cost and latency fall toward zero, and where the model runs moves from the cloud to your own perimeter to the device in your hand. No rung is the right answer on its own: the right answer is the lowest rung that clears the bar your task sets. Model type (instruct, reasoning, dense or Mixture-of-Experts) shifts cost and latency within a rung, but it is orthogonal to the tier itself.

The capability ladder: frontier → workhorse → small → edge, and the gradient of capability, cost and where-it-runs across the rungs.

Key concepts

Frontier model: The most capable tier: the largest, best-reasoning models a lab ships, at the highest cost and latency per call. Reserved for the steps that genuinely need them, not for everyday work.
Workhorse model: The mid tier that handles most production tasks at a fraction of frontier cost. The sensible default: capable enough for everyday generation, summarisation and tool use without paying for the top rung.
Small language model (SLM): A model in the roughly 1B to 15B parameter class, built for depth and repetition on a narrow task rather than broad reasoning. Fine-tunable, cheap to serve, and runnable on-prem or on modest hardware. A deployment class, not a failed large model.
Edge / on-device: Running a model directly on a laptop or phone, offline, with no API call and no per-request cost. Made possible by quantization shrinking the model to fit local memory; maximises privacy because data never leaves the device.
Quantization: Storing model weights at lower numeric precision (for example 4-bit instead of 16-bit) to shrink memory and speed inference. A 4-bit small model can drop to roughly a quarter of its full-precision footprint while keeping most of its quality, which is what lets it run on the edge.
Reasoning model: A model that spends extra inference-time compute to plan before answering, trading higher cost and latency for stronger multi-step reasoning. Contrast with an instruct model, which executes a request directly: reasoning models plan, instruct models do.
Mixture-of-Experts (MoE): An architecture where a router activates only a sparse subset of the model's parameters per token, instead of all of them as a dense model does. It gives the capacity of a large model at the compute cost of a much smaller one, shifting the cost/latency point within a tier.
Distillation: Training a smaller model to imitate a larger one, transferring much of its behaviour into a cheaper-to-run package. A common way small language models inherit strong task quality without the size or cost of the teacher model.

When to use

Good fit

Frontier when the task genuinely needs the hardest reasoning: a tricky multi-step plan, ambiguous instructions, or a one-off step where a wrong answer is expensive. Reach for it deliberately, for the step that needs it, not as the default for the whole flow.
Workhorse for everyday production work: drafting, summarising, extraction, routine tool calls. It clears the bar for most tasks at a fraction of frontier cost, which is why it should be your default and frontier the exception.
Small (SLM) for high-volume, narrow, repetitive tasks: classification, routing, tagging, structured extraction at scale, especially when data must stay on-prem. Fine-tune it on your task and it is faster, cheaper and often more accurate there than a generalist giant.
Edge / on-device when privacy is non-negotiable or the workload must run offline: a model on a laptop or phone keeps data on the device, removes per-call cost entirely, and works with no network. The right call for privacy-critical and disconnected use.

Anti-patterns

Defaulting to a frontier model for trivial, high-volume tasks. Sending classification or tagging at scale to the top rung burns roughly 10 to 20 times the cost for no quality gain the task can use. Push the volume down to the cheapest rung that clears the bar.
Dismissing small language models as toys. They are a deployment class built for depth and repetition: on a narrow fine-tuned task a small model routinely beats a generalist giant on speed and cost, and sometimes on accuracy. Judging them on broad open-ended benchmarks misses the point.
Ignoring the edge for privacy-critical or offline cases. If data cannot leave the device or the network may be absent, an API tier is the wrong tool no matter how capable: a quantized on-device model is the only rung that fits the constraint.
Picking one tier for the whole product. Tier is a per-task decision, not a company-wide one. A single flow can mix frontier, workhorse and small rungs; locking everything to one rung either overpays on the cheap tasks or underperforms on the hard ones.

Code examples

Choosing a tier by task

# A tiny sketch: pick the cheapest tier that clears the task's bar.
# The real routing logic lives in the choosing-a-model page; this is
# just the shape of the decision.

def pick_tier(task: str) -> str:
    if task in {"classify", "route", "tag", "extract"}:
        return "small"      # high-volume, narrow: SLM
    if task in {"draft", "summarise", "answer"}:
        return "workhorse"  # everyday production default
    if task in {"hard_reasoning", "plan"}:
        return "frontier"   # reserve the top rung
    return "workhorse"

# Same task, very different cost: frontier vs small runs ~10-20x apart.

The decision is per task, not per app. Send each request to the lowest rung that can do it; that is the whole cost lever, and what routing automates.

Running a small model on the edge

# A quantized small model, on your own laptop, fully offline.
# Ollama pulls a 4-bit build that fits in a couple of GB of memory.
ollama run gemma3:4b "Classify this ticket: billing, bug, or feature?"

# No API call, no per-request cost, data never leaves the machine.
# Tens of tokens per second on a recent laptop. This is the edge tier.

The edge rung in one command: a small quantized model on local hardware, offline, zero per-call cost. The tradeoff is capability, which is exactly why it suits narrow tasks.

Comparison

vs Frontier

Capability, cost, latency, where it runs

Highest capability and the best at hard reasoning, but the highest cost and latency per call, and cloud only. Use it deliberately for the steps that need it, not as a default. The wrong rung for high-volume trivial work.

vs Workhorse

Capability, cost, latency, where it runs

Capable enough for most production tasks at a fraction of frontier cost, with moderate latency, cloud-served. The sensible default tier: everyday drafting, summarising, extraction and tool use belong here.

vs Small / SLM

Capability, cost, latency, where it runs

Narrow rather than broad, but low latency and dramatically lower cost, fine-tunable, and runnable on-prem so data stays in your perimeter. Wins on high-volume repetitive tasks and in regulated settings. A class, not a consolation.

vs Edge

Capability, cost, latency, where it runs

A quantized small model on a laptop or phone: lowest capability, but offline, zero per-call cost, and maximal privacy because data never leaves the device. The only rung that fits privacy-critical or disconnected use.

vs Deploying models

A different axis: where it runs

Tier (this page) is how capable the model is; deployment is where it runs: lab API, managed inference, or self-hosted. The two axes are orthogonal: a small model can run on a managed API, a frontier model only behind a lab API. Pick the tier here, then choose the deployment on its own page.

Resources

FAQ

What is a small language model (SLM)?: A model in the roughly 1B to 15B parameter class, built for depth and repetition on a narrow task rather than broad reasoning. It is fine-tunable, cheap to serve, and small enough to run on-prem or on modest hardware. Treat it as a distinct deployment class, not a shrunken general-purpose model.
Are SLMs worse than LLMs?: Only on broad, open-ended reasoning, where a frontier model is stronger. On a narrow task it is fine-tuned for, a small model is frequently faster, far cheaper and often more accurate than a generalist giant. It trades breadth for low latency, on-prem privacy and low cost. Worse and better are the wrong frame: it is a different tool for a different job.
When should I run a model on-device?: When privacy is non-negotiable or the workload must run offline. An on-device model keeps data on the laptop or phone, removes per-call cost entirely, and needs no network. The tradeoff is capability, so it suits narrow, well-defined tasks: classification, local assistants, on-device extraction. For broad reasoning, stay on a cloud tier.
Reasoning model or instruct model?: Instruct models execute a request directly: reasoning models spend extra inference-time compute to plan first, trading higher cost and latency for stronger multi-step reasoning. Reasoning models plan, instruct models do. This is a model type, orthogonal to tier: you can have an instruct or a reasoning variant within the same capability rung.
What is Mixture-of-Experts (MoE)?: An architecture where a router activates only a sparse subset of the model parameters per token, rather than all of them as a dense model does. The result is the capacity of a large model at the compute cost of a much smaller one. Like reasoning vs instruct, it is a within-tier lever: it shifts the cost and latency point, not the tier itself.