Model right-sizing — matching the model to the task, and what the wrong one costs you

Definition

Model right-sizing is choosing the smallest model that reliably does the job, instead of defaulting to the biggest one you have access to.

The economics carry the argument. Model tiers differ severalfold in cost per token. If a task would have succeeded on a mid-tier model and you ran it on a frontier one, you paid a multiple for an identical outcome. Do that across a week of routine work and it is a substantial share of your rate-limit window, spent on nothing.

The important word is right-sizing, not down-sizing. Under-modelling is also expensive, and in a way that is easy to miss. A model that cannot quite do the task produces a wrong answer, you re-prompt, it tries again, you correct it, and three cheap attempts plus your time cost more than one adequate attempt would have. The goal is a deliberate match, not a race to the cheapest tier.

Why the default is almost always too big

Most people pick a model once and stop thinking about it. Since nobody wants to be the person whose agent failed because they were being frugal, the default that gets set is the largest available, and then it is used for everything, including work that never needed it.

The reason this persists is that over-modelling has no symptom. An oversized model on a simple task does not fail, does not warn you, and does not look any different from a right-sized one. It just quietly costs more. Under-modelling announces itself immediately, because the output is wrong. So the feedback loop is one-sided: you find out fast when the model was too small, and never when it was too big.

That asymmetry is what makes this worth measuring rather than intuiting. It is also a general pattern rather than a quirk of AI tooling. Costs that announce themselves get optimized; costs that stay silent accumulate until something external forces an audit.

What genuinely needs a frontier model

Right-sizing is not an argument for using small models everywhere. Reach for the top tier when the work involves:

• Holding a lot at once: a change that spans many files whose interactions matter, where losing the thread produces subtly wrong code rather than obviously broken code.
• Genuine ambiguity: a debugging problem where the cause is not yet known and the useful move is forming and discarding hypotheses.
• Design with consequences: decisions that are expensive to reverse later.
• Unfamiliar territory: a domain, language, or codebase where you cannot yourself tell a good answer from a plausible one, so you need the model's first answer to be right.

And the work that usually does not:

• Mechanical refactors with a clear rule.
• Boilerplate, scaffolding, config, and test fixtures.
• Renames, formatting, and moves.
• Well-specified small functions where you already know what the answer looks like.
• Reading and summarizing something.

The habit that makes it stick

Right-sizing works as a per-task question, not a global setting: do I already know what a correct answer looks like here? If yes, a smaller model is usually enough, because you can catch it being wrong. If no, you are relying on the model's judgement rather than checking it, and that is where the larger one earns its cost.

Finding your own over-modelled sessions

Because over-modelling is invisible in the moment, the only reliable way to find it is after the fact, from what you actually ran.

npx @promptster/cc-audit --judge

A bare npx @promptster/cc-audit prints your local breakdown and sends nothing. Adding --judge sends a gist of each task plus metadata, never your code, prompts, file paths, or repo names, to a hosted model that reads back which of your sessions were routed to a model well above what the work required, and what the right-sized tier would have cost instead. The flag is the consent: nothing is transmitted unless you type it.

What right-sizing is NOT

• Not "always use the cheap model": that trades token cost for retry cost and your own time, often at a loss.
• Not a fixed rule per language or repo: it is per task. The same file can warrant different tiers on different days.
• Not something you can judge from the output alone: a correct answer from an oversized model looks exactly like a correct answer from a right-sized one, which is the difficulty in a sentence.
• Not a quality compromise: on tasks that did not need the larger model, the smaller one produces the same result for less.

Related reading

• Rate-limit window — what over-modelling is spending.
• Context compaction — the other habit that quietly drives cost.
• Context efficiency — carrying less, which compounds with choosing right.