Short answer: Context engineering is choosing what an AI model sees at each step of a task. Anthropic defines it as "the set of strategies for curating and maintaining the optimal set of tokens" during inference, including information beyond the prompt.
It matters because in long agent runs, a missing or buried fact can break a step that a well-worded prompt cannot save. If you build or buy AI agents, this is the skill that decides whether a long run still works at its later steps.
How context engineering works: the model sees far more than your prompt
Context engineering works by deciding, before each model call, which of five layers go into the context window. Your typed prompt is only one of them.
- System instructions: who the model is, which rules it follows and what format it answers in. This used to be the whole job of prompt engineering.
- Memory: what the system keeps from earlier in the session or from past sessions. Our guide to how AI agent memory works covers the storage side.
- Retrieved context: documents or records pulled in for this task, usually through retrieval-augmented generation (RAG).
- Tool definitions: the functions the model can call and what each one does, explained in our piece on function calling and tool use.
- Conversation history: everything said so far that is still relevant.
Shopify CEO Tobi Lutke described the idea in June 2025 as "the art of providing all the context for the task to be plausibly solvable by the LLM." Andrej Karpathy called it "the delicate art and science of filling the context window."
Prompt engineering vs context engineering: one input versus everything else
Prompt engineering is a subset of context engineering. Anthropic describes prompt engineering as "methods for writing and organizing LLM instructions for optimal outcomes," while context engineering covers every token the model sees.
| Prompt engineering | Context engineering | |
|---|---|---|
| Main question | How should I phrase this? | What does the model need to know right now? |
| What it manages | The instruction itself | Instructions, memory, retrieved data, tools, history |
| Where it matters most | A single, one-off question | Long, multi-step agent workflows |
| Typical failure | A vague or ambiguous instruction | The right fact missing, stale or buried at that step |
| Still needed? | Yes, as one layer of the context | Yes, and it grows with every step an agent takes |
For a one-shot question, the two look the same. Deep into an agent run, the first instruction's wording matters less. What counts is what the agent still remembers and what it retrieved.
Vendors frame the shift as urgent. DataHub's 2026 survey of 250 IT and data leaders says 82% agree "prompt engineering alone is no longer sufficient to power AI at scale." Treat that as opinion: it is self-reported, and DataHub sells a context management platform.
More context is not better, because recall falls as inputs grow
Adding tokens has diminishing returns. Anthropic's guide to effective context engineering for AI agents says that "as the number of tokens in the context window increases, the model's ability to accurately recall information" declines. It treats context as a limited resource. Practitioners call this recall decline context rot.
Position matters too. The Lost in the Middle paper found that "performance can degrade significantly when changing the position of relevant information." Its authors add that performance "is often highest when relevant information occurs at the beginning or end of the input context."
The practical consequence is simple: the skill is curating the window, not filling it.
Try it yourself: run a per-step budget of keep, retrieve, compact, reset
Run this budget before each step of an agent or long chat. Each move maps to Anthropic's guide or the Lost in the Middle finding.

- Keep: leave in the window only the instructions, tool definitions and history this step uses. Drop old tool results; Anthropic advises clearing them.
- Retrieve: fetch documents when the step needs them, instead of pasting everything up front. Put the key fact near the start or end of the input.
- Compact: when a conversation nears the window limit, summarize it. Anthropic defines compaction as "taking a conversation nearing the context window limit, summarizing its contents."
- Reset: hand a focused job to a sub-agent. Anthropic notes that "specialized sub-agents can handle focused tasks with clean context windows."
Between steps, persist what must survive. In Anthropic's notes pattern, "the agent regularly writes notes persisted to memory outside of the context window." To see what actually reached the model at each step, use tracing from AI agent observability tools.
Decision rule: if a task fits in one exchange, a clear prompt is enough. If it spans many steps, tools or documents, budget the context at every step.





