# Context Window: How Much an AI Can Read at Once

> What an AI model's context window is, what takes up space in it, how big it is today, and why fitting something in does not mean the model will use it.

*Tags: ai, fundamentals, context, ai-design*

---


> [!info] Quick Definition
> The **context window** is the maximum number of [[tokens|tokens]] a model can handle in a single request. It includes the instructions, the conversation history, the documents and tools you pass in, and also the response the model generates.


## What is a context window?

A language model remembers nothing between one call and the next. Everything it needs to answer has to travel in the request: the system message, earlier turns, attached files and the current question. The context window is the maximum size of that package.

Anthropic's documentation describes it as the model's "working memory". It is not the same as the training data, which is fixed and has a cutoff date (see [[knowledge-cutoff|knowledge cutoff]]).

When an assistant seems to "remember" what you said ten messages ago, it is because the product resends the whole history on every turn. That is why a conversation takes up more and more space, and why each turn costs a little more than the one before.

## What takes up space in the window?

More than you might think. According to Claude's documentation, all of this counts:

- **The system prompt**, which is repeated in every call. See [[system-prompt|system prompt]].
- **Every message**, including tool results, images and documents.
- **Tool definitions.** An agent with many connected tools spends tokens before the user types anything. See [[tool-use|tool use]].
- **The model's reply**, including its reasoning ("thinking") tokens. If the model needs to write 4,000 tokens, those tokens have to fit in the same window. See [[reasoning-models|reasoning models]].

[[prompt-caching|Prompt caching]] changes what you pay for a repeated block, but not the space it takes: cached content still counts toward the window.

What if you go over? In the Claude API, if the input alone exceeds the window, the request fails with a 400 error ("prompt is too long"). If the reply hits the limit while it is being generated, Claude 4.5 and later models stop with the reason `model_context_window_exceeded`. The user sees an error or a half-finished answer, and the team has to design what appears in that case.

## How big are context windows today?

The figures change with every model generation. These are the ones the three major providers publish in their official documentation, checked on October 18, 2026:

| Model | Context window | Max output per request |
|---|---|---|
| Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 (Anthropic) | 1,000,000 tokens | 128,000 tokens |
| Claude Sonnet 4.5 and other earlier models (Anthropic) | 200,000 tokens | Depends on the model |
| GPT-6 Astra (OpenAI) | 1,050,000 tokens | 128,000 tokens |
| Gemini 4 Argon (Google) | 1,048,576 tokens, shared across input, reasoning and output | 65,536 tokens by default |

According to Anthropic, 1 million tokens is about 555,000 words with the tokenizer it has used since Claude Opus 4.7. Fewer Spanish words fit in the same window, because each word usually takes more tokens.

Three nuances matter when you design:

- **Not everyone counts the same way.** In Gemini 4 Argon, input and output share the same million; in Gemini 3 and 2.5 they were separate budgets.
- **There are other limits.** Claude accepts at most 600 images or PDF pages per request, so a large batch of files can hit that before it hits the token limit.
- **Your product may use less than the model.** What matters is the limit of your integration, not the number in the announcement. Check the current documentation before designing a flow that depends on a specific size.

## Does fitting something in mean the model uses it?

No. The "Lost in the Middle" study (Liu et al., 2023, published in TACL) found that models perform best when the relevant information is at the beginning or end of a long context, and much worse when it is in the middle. It used 2023 models, but the problem persisted as windows grew: in July 2025, Chroma tested 18 models, including GPT-4.1, Claude 4 and Gemini 2.5, and concluded in its "Context Rot" report that performance becomes less reliable as input grows, even on simple tasks. Anthropic's documentation uses the same term: as token count grows, accuracy and recall degrade.

Google says something similar: Gemini is very good at finding a single fact in a long document (the "needle in a haystack" test), but when it has to retrieve several facts at once, accuracy drops and varies widely.

What to do about it:

1. Put the question at the end, after the documents, as Google recommends.
2. Include only what the task needs; every irrelevant block competes for the model's attention.
3. Test with documents as long as the ones your users will use, and deliberately look for facts in the middle of the text.

## Why does this matter to a designer?

Because the window defines what your product can "remember" within a session, and that shows up in the interface:

- **Long conversations.** If the user works with the assistant for an hour, the history grows. Someone has to decide what gets summarized, what gets dropped, and whether the user finds out. See [[conversation-history|conversation history]].
- **Large documents.** An 80-page contract fitting in the window does not mean the model pays equal attention to every part of it.
- **Agents.** An agent piles up tool results at every step and can fill the window in a single task. See [[context-engineering|context engineering]] and [[agent-memory|agent memory]].
- **Cost and speed.** Every token in the window is paid for and processed on every turn. See [[cost-per-token|cost per token]] and [[llm-latency|LLM latency]].
- **Transparency.** If you trim context without telling the user, they will see answers that ignore things they said earlier. That is a trust problem, not only a technical one.

Anthropic offers **compaction** in beta: it replaces older turns with a summary the model writes, so the conversation can continue without leaving the window. Because this changes what the model "knows", the interface should say so: a notice such as "I summarized the first part of the conversation" explains why an old detail no longer shows up.

## When NOT to fill the window?

Filling it out of habit costs more, takes longer and, as we saw, dilutes the signal. To find information across many documents, it is usually better to [[rag|retrieve only the relevant fragments]] than to pass the whole corpus. Passing everything makes sense when the task requires reading the full document (summarizing a contract, comparing two versions of a spec).

## Example

Illustrative example: an assistant answers questions about a design system with 300 components whose documentation takes about 450,000 tokens.

- **Include the whole catalog in every request.** It fits in a 1-million-token window, but not in a 200,000-token one. And for each question almost the entire catalog is noise, exactly the scenario where performance degrades.
- **Search first and pass only what is related.** The product retrieves the 12 most relevant components, about 15,000 tokens: a request 30 times smaller, cheaper and faster.

The second option requires a design decision. A note such as "12 of 300 components were considered", with a link to see which ones, keeps the user from assuming the answer covers the whole system.

## Common mistakes

- Believing a large window removes the need to choose what information goes in.
- Designing around the number in the model announcement instead of your integration's real limit.
- Forgetting that the reply, the reasoning and the tool definitions also take up the window.
- Trimming or summarizing history without telling the user or keeping the key instructions.
- Putting the question first and 100 pages of documents after it.

## Sources

- [Context windows (Claude documentation)](https://platform.claude.com/docs/en/build-with-claude/context-windows)
- [Models overview (Claude documentation)](https://platform.claude.com/docs/en/about-claude/models/overview)
- [Compaction overview (Claude documentation)](https://platform.claude.com/docs/en/build-with-claude/compaction)
- [GPT-6 Astra (OpenAI documentation)](https://developers.openai.com/api/docs/models/gpt-6-astra)
- [Gemini 4 Argon (Gemini API documentation)](https://ai.google.dev/gemini-api/docs/models/gemini-4-argon)
- [Long context (Gemini API documentation)](https://ai.google.dev/gemini-api/docs/long-context)
- [Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172)](https://arxiv.org/abs/2307.03172)
- [Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma, 2025)](https://www.trychroma.com/research/context-rot)


---

Source: https://www.fernandoux.com/en/wiki/ai/context-window/