Context Engineering: Beyond the Prompt
Context engineering decides which tokens the model sees at each step: instructions, tools, history and data. Why it goes beyond the prompt and how to apply it.
What is it, concretely?
In its September 29, 2025 engineering article, Anthropic describes context engineering as the natural evolution of prompt engineering: where prompt engineering is about writing and organizing instructions, context engineering curates and maintains the optimal set of tokens during inference, including information beyond the prompt. It is also not a one-off task: it repeats every time content is passed to the model. The question shifts from “how do I phrase the instruction” to “what should be visible to the model right now, and what should not”.
In a simple product, the context is made of the system message, the history and the user’s message. In an agent, it grows with every step: tool definitions, search results, files read and summaries of work already done. Each piece takes space in the context window and competes with the others for the model’s attention.
According to the same publication, good context has these traits: the system prompt says the minimum necessary at a balanced level of specificity; tools are token-efficient and do not overlap; examples are few, diverse and representative rather than a list of rules for edge cases; and the history is informative but lean. Anthropic sums up the criterion in one line: find the smallest possible set of high-signal tokens that leads to the desired outcome.
Why isn’t more context always better?
According to Anthropic, context is a finite resource with diminishing returns. As the window fills, the model’s ability to accurately recall what is in it drops, a pattern the article calls context rot. It shows up in all models, though at different rates, and every added token draws down part of the model’s “attention budget”.
An earlier academic study points the same way. In “Lost in the Middle” (Liu et al., 2023), the models tested performed best when the relevant information was at the beginning or end of a long context and worse when it was in the middle, even for models built for long contexts. This is a 2023 result on models of that time, so do not treat it as a fixed rule for current models. The design lesson is different: putting everything in the window is not free, and selection and order matter.
Why does this matter to a designer?
Because context is what the product “remembers” and “knows” at a given moment, and that can be designed:
- What the user sees of what the model knows. If the agent consulted three sources, the interface can show which ones.
- When something is forgotten. If context is compacted, some things disappear. Users should be able to review and restore what matters.
- How much information enters. More information does not always mean a better answer. An irrelevant fragment can worsen the result.
- Continuity across sessions. What is stored outside the window, such as progress notes, defines what the agent can resume tomorrow. See memory in agents.
What techniques exist for long tasks?
Anthropic mentions three techniques:
| Technique | What it does | When it works best |
|---|---|---|
| Compaction | Summarizes a nearly full context and restarts with the summary | Extended conversations or tasks |
| Structured notes | Stores progress outside the window and reloads it later | Milestone-driven work |
| Sub-agents | A specialized agent works in its own context and returns a summary | Parallel research |
Compaction can be as simple as clearing old tool results, which is its lightest form. With sub-agents, each one works in a clean window and returns a condensed summary to the main agent, which the article says is often 1,000 to 2,000 tokens. Anthropic suggests choosing by task: compaction for flowing conversations, notes for iterative milestone work and sub-agents for parallel research.
It also recommends just-in-time retrieval: instead of preloading all data, the agent keeps lightweight references, such as file paths or queries, and loads content when needed with a tool. For less dynamic domains, such as legal or finance work, the article allows a hybrid approach: some data is loaded up front and the agent explores the rest.
When NOT to use it?
When the task is short and fits entirely in one prompt. Setting up compaction, notes and sub-agents for a one-line request adds complexity without benefit; the article itself advises doing the simplest thing that works. Nor is it wise if you do not measure what happens to the context: without observing what is loaded and how the answer changes, you cannot know whether you are improving.
How do you apply it in your product?
- Inventory the context. List everything that enters a typical request: instructions, tools, history, documents, previous results. Measure how many tokens each block takes.
- Remove redundancy. If two tools do nearly the same thing, or two instructions say the same thing in different words, keep one.
- Load on demand. Move anything used only in specific cases into a tool.
- Define what survives compaction. Write a list of information that must never be lost, such as user decisions or constraints.
- Measure the effect. Compare quality and cost before and after each change, on a set of real cases.
Example
Illustrative example: a user research agent reads 40 interviews to produce a report. At first it loads every transcript, and the context fills before reaching the midpoint. The team changes the approach: the agent creates one note per interview with three findings, stores the list of themes in a progress file, and loads only the notes relevant to each section of the report. Report quality improves, and the designer can see in the interface which interviews were used in each section.
Common mistakes
- Thinking context engineering is just a longer prompt. A bloated system prompt also spends attention.
- Filling the window with tools that overlap.
- Compacting without reviewing what information gets lost.
- Not measuring how the answer changes when the context changes.
- Keeping progress in the chat instead of in a place the agent can reconsult.