Small vs. Large Models: When to Use Each
A large model reasons better; a small one answers sooner and costs less. How to choose by task, with Anthropic, OpenAI and Google tiers and your own tests.
What do “large” and “small” mean?
A model’s size is measured mainly by its number of parameters, the values adjusted during training. More parameters allow capturing more patterns, but they require more memory and more compute for each answer.
The relationship between size, data and compute has been studied formally. Kaplan and colleagues (2020) found that a language model’s loss falls following a power law with model size, dataset size and training compute, and that larger models make better use of their data. Two years later, the Chinchilla paper (Hoffmann and colleagues, 2022) concluded that the models of the time were significantly undertrained: every time model size doubles, training tokens should double too. Its 70-billion-parameter model used the same compute as Gopher, at 280 billion, but four times more data, and outperformed it on a wide range of tasks.
The lesson for you is twofold: size matters, but it is not the only factor, and a smaller, well-trained model can compete with a larger one.
In real products, “small” and “large” are almost never a parameter count: they are tiers from the same provider. This is what each one says on its official pages (consulted on November 2, 2026, summarized description):
| Provider | Lower tier | Higher tier |
|---|---|---|
| Anthropic | Haiku 5.5: the fastest in the family, for high-volume, latency-sensitive tasks such as classification, extraction and routing | Fable 5.1: for demanding reasoning and long-horizon agentic work. Sonnet 5.5 sits in the middle, as the best combination of speed and intelligence |
| OpenAI | gpt-6-luna: the most efficient, for focused, high-volume tasks | gpt-6-astra: the most capable, for the most demanding work. gpt-6.1-sol promises near-Astra performance at a lower cost |
| Gemini 3.5 Flash-Lite: the fastest and most cost-effective in the 3.5 family | Gemini 3.8 Flash: its most intelligent Flash model (the page consulted does not describe a current Pro) |
Two useful details. The pages consulted do not publish the parameter counts of these models, so you cannot compare by real size. And the tier does not determine the context: according to those pages, Haiku 5.5, Sonnet 5.5 and Fable 5.1 all have a 1-million-token window, and so do the three gpt-6 models in the table, at 1.05 million. For what that window is for, see context window.
Why does this matter to a designer?
Because the tier you pick changes the experience in four visible ways:
- Speed. Anthropic labels Haiku 5.5 as the fastest in its lineup and Fable 5.1 as slower, while noting that real latency depends on prompt length, output length and reasoning effort. For autocomplete or classification, that difference matters more than sophistication. See latency and tokens per second.
- Cost. Between Haiku 5.5 and Fable 5.1 there is a factor of 100 in the input price: 0.10 versus 10 dollars per million tokens, according to Anthropic’s pricing page consulted on November 2, 2026. Every provider’s rates are in cost per token.
- Perceived quality. A large model handles nuance and complex instructions better. If your interface depends on that, the small one will show.
- Privacy and deployment. Small models are easier to run on your own infrastructure. See running AI locally and open vs. closed models.
When should you NOT use a small model?
When the task requires multi-step reasoning, a lot of information at once, or tone and cultural nuance that the user will judge. A tool that drafts sensitive replies to customers may need the most capable model. Nor is it a good idea if the team cannot measure the difference: without evaluation, you do not know whether the saving costs quality.
How do you decide between them?
Anthropic describes two starting points. The first is efficiency first: you start with Haiku 5.5, test thoroughly and move up a tier only for a specific gap; it suits prototypes, tight latency and simple high-volume tasks. The second is capability first: you start with a strong model for complex tasks and optimize toward more efficient models later. In both cases, its guide stresses that the most important step is a good evaluation set of your own.
A practical method:
- Define the task and the success criterion with real examples.
- Try the small model first, on a set of cases.
- If it fails on a specific type of case, try the large one only for that type.
- Measure quality, latency and cost per answer.
- Before changing tier, try the reasoning effort. Anthropic notes that tuning it is often a better lever than switching models, and OpenAI documents that gpt-6-luna supports effort levels from none to max. See reasoning models.
Can you combine a small and a large model?
Yes, but carefully. In its internal tests, Anthropic found that delegating work to cheaper models saved money in only two situations: routine work with a long cost tail, and work larger than one context window. With an Opus 5.5 orchestrator and Haiku 5.5 subagents on a coding task, cost per attempt fell 32%, but the pass rate also fell by about 4 percentage points; that was equivalent to what you get by lowering effort without subagents. When the work is a single chain of dependent steps, or fits in one context, the main model alone at lower effort came out ahead in every case measured. These are internal, directional figures: measure on your own workload. The patterns are explained in orchestrator and worker and subagents.
Example
Illustrative example: a design tool tags user comments into five categories, a million times a month. Each request carries about 500 input tokens and 20 output tokens (assumed figures). With the prices on Anthropic’s page of November 2, 2026, with no caching or batching, the monthly cost would be about 60 dollars with Haiku 5.5, 1,200 with Sonnet 5.5 and 6,000 with Fable 5.1.
The team tests all three on 200 real comments already tagged by hand. Suppose Haiku is right 90% of the time and Sonnet 92%. If users do not notice those two points, the team tags in real time with Haiku and keeps the larger model for the weekly summary, where writing quality matters more. The percentages are hypothetical; what is useful is the method: compute the cost with real prices and measure quality before deciding.
Common mistakes
- Choosing the largest model “just in case” without trying the small one.
- Comparing models only on published general results when your task is specific.
- Assuming cost depends only on the tier: input and output length count too, and Haiku 5.5 charges more when the prompt exceeds 100,000 tokens.
- Delegating to cheap models without measuring how much quality is lost.
- Switching models without re-evaluating the whole experience.
Sources
- Hoffmann et al., “Training Compute-Optimal Large Language Models” (2022): https://arxiv.org/abs/2203.15556
- Kaplan et al., “Scaling Laws for Neural Language Models” (2020): https://arxiv.org/abs/2001.08361
- Anthropic, “Models overview” (consulted on November 2, 2026): https://platform.claude.com/docs/en/about-claude/models/overview
- Anthropic, “Choosing the right model” (starting approaches and effort): https://platform.claude.com/docs/en/about-claude/models/choosing-a-model
- Anthropic, “Optimizing for cost and intelligence” (delegation): https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence
- Anthropic, “Pricing” (consulted on November 2, 2026): https://platform.claude.com/docs/en/about-claude/pricing
- OpenAI, “Models” (consulted on November 2, 2026): https://developers.openai.com/api/docs/models
- OpenAI, “GPT-6 Luna” (effort): https://developers.openai.com/api/docs/models/gpt-6-luna
- Google, “Gemini models” (consulted on November 2, 2026): https://ai.google.dev/gemini-api/docs/models