Should I just use the most powerful model?

No. The most capable model is also the slowest and most expensive, so it is the wrong default for most of what a production system does. Capability is one variable among several: a routing task or a high-volume classification does not get better answers from a frontier model, it just costs more. Reserve the top tier for the steps that genuinely need it and route the rest to a cheaper tier that clears the bar.

How do I pick a benchmark?

Match the benchmark to the task and the tier. Use a coding benchmark like SWE-bench for a coding task, an agentic suite for tool-using agents, and a reasoning leaderboard only for genuine reasoning work. Do not score a routing or classification task on frontier reasoning leaderboards. Treat any benchmark as an entry point that shortlists candidates, never as the verdict: that comes from your own evals.

Why run my own evals?

Because public benchmarks never match your task distribution. A model that tops a leaderboard can still fail on your inputs, your formats and your edge cases. Your own eval is a golden set of a few dozen real cases with known-good outputs, scored on the full trace for agents, not just the final answer. Benchmarks shortlist, evals decide: the eval is the selection gate, not an optional extra.

What is model routing?

A layer that reads each request, estimates its difficulty and dispatches it to the cheapest tier that can handle it: routine calls to a small model, hard reasoning to a frontier model. The router itself is a small, fast model whose overhead is negligible against the frontier call it avoids. It typically cuts cost by more than half with no quality loss, because most requests never needed the expensive model. Routing is the output of the selection process, not a detail.

How do I choose a model for an agent?

Select on agent-specific signals, not raw reasoning score. The ones that matter are tool-calling reliability and low schema hallucination (does it invent parameters that are not in the schema?), structured-output consistency on nested JSON, goal coherence across a long sequence of steps, and cached-input economics as the tool trace grows. Run trace-based evals on a golden set, scoring every tool call and not just the final answer: the trace is where agent failures hide.

← Stack

Choosing a model

Choosing a model is a process, not a pick: five steps that end in a routing strategy, not a single winner.

Overview

The instinct is to pick the most powerful model and move on. That instinct is wrong. Raw capability is one variable among several, and the most capable model is also the slowest and the most expensive, which makes it the wrong default for most of what a production system actually does. Choosing well is not a pick, it is a process: a repeatable decision you can defend, rerun when a new model ships, and apply per task rather than once for the whole company. The page below lays that process out as five steps, in order, because the order is the point: you eliminate before you compare, and you classify before you benchmark.

The five steps are: eliminate candidates that fail a hard constraint (data privacy and residency, a latency ceiling, a budget, a license) before quality is even on the table; classify the task by intelligence tier, frontier, workhorse or small, so you benchmark a task against the right peers; read the benchmarks that match that tier, treating leaderboards as entry points rather than verdicts; shortlist three to five candidates and run your own evals on your own task distribution; and design a routing strategy instead of betting everything on one model. Each step narrows the field, and the order keeps you from wasting an eval budget on a model that was never going to clear a non-negotiable.

The payoff is the fifth step, and it is an architecture decision, not a detail. The output of the process is not "model X won", it is a routing strategy: a cheap router model reads each request, decides which tier it needs, and dispatches it there, so the frontier model is reserved for the steps that genuinely need it. This is where model != provider pays off: code against an interface and choose a model per node, and you can swap one model for another without touching the system around it. Routing typically cuts cost by more than half with no quality loss, because most requests never needed the expensive model in the first place. The model choice is a system design choice, which is why this page sits above the rest of the cluster.

Architecture

Read the funnel top to bottom. You start with many candidate models and narrow at each step: non-negotiables remove anything that fails a hard constraint, tier classification fixes the peer group, the right benchmarks shortlist within that group, your own evals rank the shortlist on your task, and routing turns the survivors into a strategy. Notice that the funnel does not end in a single model: it ends in a routing strategy, a small router dispatching each request to the tier it needs. Each stage is cheaper to run than the one above it is to skip, which is why the order holds.

The selection funnel: many candidate models narrow through non-negotiables, tier, benchmarks and your own evals, ending not in one winner but in a routing strategy.

Key concepts

Non-negotiable (hard constraint): A pass or fail requirement that eliminates a model before quality is considered: data residency or sovereignty, a latency ceiling, a budget cap, a license term. The four that recur are data privacy, latency, cost and required intelligence. If a model fails one, no benchmark score saves it.
Intelligence tier: The capability class a task belongs to: frontier, workhorse or small. Classifying the task first is what lets you benchmark it against the right peers rather than scoring a routing task on frontier reasoning leaderboards. The tier ladder has its own page.
Benchmark vs eval: A benchmark is a public, shared test (a leaderboard, SWE-bench, an agentic suite) that ranks models on a generic distribution. An eval is your own test on your own task distribution. Benchmarks shortlist; evals decide. They answer different questions, and only the eval reflects your actual workload.
Golden set: A small, curated set of representative inputs with known-good expected outputs, drawn from real cases and real failures. It is the ground truth your evals score against: a few dozen well-chosen examples beat a generic benchmark for selecting a model on your task.
Trace-based eval: Scoring an agent on its full execution trace, every tool call, argument and intermediate step, not just the final answer. Tracing tells you what happened; evaluation tells you whether it was correct. For agents, the trace is where most failures hide, so it is where the eval has to look.
Model routing: A layer that estimates each request difficulty and dispatches it to the cheapest tier that can handle it: routine calls to a small model, hard reasoning to a frontier model. It is the output of the selection process, not a detail: routing typically cuts cost by more than half with no quality loss.
Shadow router: A small, fast model whose only job is to read a request and decide which tier it needs, usually in well under the time the real call takes. It is cheap enough that its overhead is negligible against the frontier call it avoids, which is what makes routing pay.
Tool-calling reliability: How consistently a model calls the right tool, at the right time, with arguments that match the schema, without inventing parameters or fields that do not exist. The risk rises with tool count, and it is a primary selection signal for an agent model, separate from raw reasoning score.
Prompt caching: Reusing the model computation for a repeated prefix (a system prompt, a tool schema, a long context) so you are not billed full price for it on every call. It matters most in agent loops, where the same context is resent on every step and tool traces grow: cached-input pricing can dominate the bill.

When to use

Good fit

Production systems that run at volume. When a flow handles thousands of requests, the gap between the cheapest tier that clears the bar and a frontier default compounds fast. The full process pays for itself the moment routing sends the routine majority to a small model and reserves frontier for the few steps that need it.
Cost-sensitive scale, where the model bill is a line item someone watches. Here the routing strategy is the whole point: a small router dispatching by tier typically cuts cost by more than half with no quality loss, because most requests never needed the expensive model. The process turns that saving into a defensible design rather than a guess.
Regulated or sovereign data, where the first step decides everything. When data cannot leave a jurisdiction or a perimeter, the non-negotiable eliminates whole classes of model before quality is considered. Running the process in order keeps you from falling for a benchmark winner you were never allowed to deploy.
Agentic workloads with tools and long horizons. An agent that calls tools over many steps is selected on different signals than a chatbot: tool-calling reliability, structured-output consistency on nested JSON, goal coherence across a long trace, and cached-input economics as the trace grows. Trace-based evals on a golden set are the only honest way to compare candidates here.

Anti-patterns

Picking by leaderboard rank alone. A top-line benchmark score ranks models on a generic distribution that is not yours. Reaching for the number-one model without checking it against your constraints and your task is the single most common selection mistake: the leaderboard is an entry point, not a verdict.
Betting everything on one frontier model. A single expensive model for every request overpays on the routine majority and leaves you with no fallback when it is slow, down or repriced. The process ends in a routing strategy precisely so you are never one model deep.
Skipping your own evals. Public benchmarks never match your task distribution, so a model that tops them can still fail on your inputs. Shipping a model you have not evaluated on a golden set drawn from your real cases is choosing blind: the eval is the selection gate, not an optional extra.
Over-routing a trivial app. Routing is an architecture decision, and like any architecture it has a cost. A small app with one task and low volume does not need a router and a multi-tier strategy: it needs one well-chosen workhorse model. Run the process, but let it conclude with a single model when that is the honest answer.

Code examples

A task-to-tier router

# Step 5 in code: a cheap router reads the request and dispatches it
# to the tier it needs. Code against an interface, choose a model per
# node, so swapping a model never touches the system around it.

def route(request) -> str:
    difficulty = classify(request)        # a small, fast shadow router
    if difficulty == "hard":
        return "frontier"                 # reserve for steps that need it
    if difficulty == "routine":
        return "workhorse"                # the everyday default
    return "small"                        # high-volume, narrow tasks

# Most requests are not "hard", so most never hit the frontier model.
# That is the saving: routing typically cuts cost 60%+, no quality loss.

The router itself is a small model, cheap enough that its overhead is negligible against the frontier call it avoids. The output of selection is this dispatch, not a single winning model.

Scoring a shortlist on a golden set

# Step 4 in code: rank a shortlist on YOUR task, not a public benchmark.
# A golden set is a few dozen real cases with known-good outputs.

golden = load_golden_set()                # real inputs, expected outputs

def evaluate(model) -> float:
    passed = 0
    for case in golden:
        trace = run_agent(model, case.input)   # full trace, not just output
        if score_trace(trace, case.expected):  # tool calls + final answer
            passed += 1
    return passed / len(golden)

shortlist = ["candidate_a", "candidate_b", "candidate_c"]
ranked = sorted(shortlist, key=evaluate, reverse=True)

Benchmarks shortlist, evals decide. Scoring the full trace (every tool call, not just the final answer) is what catches the failures that matter for an agent. The golden set is your ground truth.

Comparison

vs Model tiers

How this relates: the tiers step 2 classifies into

This page is the process; model tiers is the ladder the process classifies tasks into. Step 2 maps each task to frontier, workhorse or small, and the tiers page defines what those rungs are and when each wins. Read the tiers page to understand the rungs, read this one to run the decision.

vs Deploying models

How this relates: where the chosen model runs

Choosing a model decides which model; deploying models decides where it runs: a lab API, managed inference, or self-hosted. The two are orthogonal and sequential: run this process to pick the model and tier, then choose the deployment on its own page. Sovereignty as a non-negotiable here often points straight to a deployment answer there.

Resources

FAQ

Should I just use the most powerful model?: No. The most capable model is also the slowest and most expensive, so it is the wrong default for most of what a production system does. Capability is one variable among several: a routing task or a high-volume classification does not get better answers from a frontier model, it just costs more. Reserve the top tier for the steps that genuinely need it and route the rest to a cheaper tier that clears the bar.
How do I pick a benchmark?: Match the benchmark to the task and the tier. Use a coding benchmark like SWE-bench for a coding task, an agentic suite for tool-using agents, and a reasoning leaderboard only for genuine reasoning work. Do not score a routing or classification task on frontier reasoning leaderboards. Treat any benchmark as an entry point that shortlists candidates, never as the verdict: that comes from your own evals.
Why run my own evals?: Because public benchmarks never match your task distribution. A model that tops a leaderboard can still fail on your inputs, your formats and your edge cases. Your own eval is a golden set of a few dozen real cases with known-good outputs, scored on the full trace for agents, not just the final answer. Benchmarks shortlist, evals decide: the eval is the selection gate, not an optional extra.
What is model routing?: A layer that reads each request, estimates its difficulty and dispatches it to the cheapest tier that can handle it: routine calls to a small model, hard reasoning to a frontier model. The router itself is a small, fast model whose overhead is negligible against the frontier call it avoids. It typically cuts cost by more than half with no quality loss, because most requests never needed the expensive model. Routing is the output of the selection process, not a detail.
How do I choose a model for an agent?: Select on agent-specific signals, not raw reasoning score. The ones that matter are tool-calling reliability and low schema hallucination (does it invent parameters that are not in the schema?), structured-output consistency on nested JSON, goal coherence across a long sequence of steps, and cached-input economics as the tool trace grows. Run trace-based evals on a golden set, scoring every tool call and not just the final answer: the trace is where agent failures hide.