Context Engineering
The agent loop rebuilds context each turn. Instructions, tool definitions, conversation history, file reads, search results, and your current task — everything enters the same window, and the model predicts from all of it. What enters, where it lands, and when it expires determines whether the model can do useful work.
Context engineering is the practice of shaping each LLM call's context to be optimal for its task. It applies to every call — not just when the window is full. A fresh session with a clear task benefits from the same thinking as a 200K-token conversation approaching compaction.
This chapter covers the five mechanisms that control what enters the context window and where: context files, MCP tool schemas, skills, sub-agents, and context compaction. Each has different tradeoffs between token cost and attention reliability. The sections that follow start with the fundamental constraint — why position in the window matters — then work through each mechanism.
The Attention Curve: Why Placement Matters
Context management
Attention is strongest at the edges
Drag the slider to 50%. The diagram is normalized: 50% means halfway through the window, whether the window holds 10K, 100K, or 1M tokens. The same failure exists at every size. What changes is the scale of the weak middle. At 50% fill, those windows contain 5K, 50K, and 500K tokens respectively. The model must find relevant information amid ever more competing context. A larger window gives the agent more room; it does not give the agent uniform recall across that room.
Three zones emerge from the curve. The primacy zone at the start — where system instructions, user rules, and project files land. The recency zone at the end — where your current task and recent tool results sit. And the dead center — the middle of the window, where attention drops off sharply.
Why does this happen? Recall that attention is relevance-weighted context mixing, not memory. For each next token, a causal transformer builds a new weighted mix of the tokens that came before it. That mix is not position-neutral.
The two edges have structural advantages. Opening tokens are visible to every later computation, and models often turn some of them into attention sinks. The newest tokens are closest to the prediction frontier, where trained models tend to give recent context more weight. Middle tokens get neither advantage: they must compete with the opening context's accumulated influence and the end of the prompt's recency bias.
Causal attention, positional encoding, and learned attention patterns determine how strong each advantage is. The curve differs by model, task, and context length, but the engineering consequence is consistent: information placed in the middle has a lower chance of shaping the next prediction. Initial tokens can receive disproportionately high attention even when they carry no semantic importance.1
This is not a single bug to fix. The lost-in-the-middle effect means the same instruction can be reliable at the top or bottom of the window and unreliable in the middle.2
Now drag the slider toward high utilization. The J-curve is a long-context failure mode, not a universal phase change at 70% fill.
At modest absolute lengths, both primacy and recency can remain useful: the curve is U-shaped. As the sequence becomes much longer, primacy can erode while recency remains strong, turning the U into a J and expanding the unreliable middle. The transition depends on the model, task, and absolute token count — not a fixed utilization percentage. A 10K context at 80% fill contains 8K tokens; a 1M context at the same fill contains 800K. The slider normalizes position, so it cannot predict where a particular model's J-curve begins.
Recent evaluations show that this is not an artifact of an older model generation. Du et al. tested five open and closed models on coding, question-answering, and math tasks: performance fell 13.9–85% as input length grew within the models' claimed windows, even with perfect retrieval and masked or whitespace distractors.3 Chroma's Context Rot tested 18 frontier models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3; every model showed non-uniform degradation as input length grew, with coherent and semantically similar distractors making the decline worse.4 This is the constraint that context engineering works within.
Every mechanism in this chapter — context files, tool schemas, skills, sub-agents, compaction — is a way to keep important information where the model can still attend to it. The job is not to fit everything. The job is to keep the active task, current constraints, and relevant evidence in the high-attention zones.
The Context Pressure Problem
The agent rebuilds context each turn: a fixed prefix (system instructions, tool definitions, context files) followed by everything appended so far. When you type your task, it lands at the end — in the recency zone, where attention is strongest. Then the agent starts working. Tool calls, tool results, file reads, search hits — each one appends after your prompt.
Your prompt does not move. The context grows around it. After a few turns, tens of thousands of tokens sit between your task and the end of the window. It is no longer in the recency zone. It is in the middle — the zone the attention curve showed as weakest.
This is the context pressure problem: the more work the agent does, the further your original instructions drift from where the model can still attend to them.
Context management
Context pressure moves the task through the window
Try the presets. Fresh Session keeps the window light — the task gets strong attention. Heavy MCP (eager) shows the real danger: connecting a few MCP servers plus conversation pushes the task into the dead zone, even though the window isn't full. Heavy MCP (deferred) demonstrates the dramatic savings from deferred tool loading — same tool count, fraction of the tokens. Deep Conversation shows how long conversations alone create pressure — 15 turns of accumulated context push the task down. Skill-Heavy shows procedure expansions: turns where a skill or reusable workflow was invoked get a violet accent bar, and their extra 4K tokens per expansion add up fast. Near Compaction shows the drain cascade in action — the hatched region models capacity reserved for the compaction handoff, shrinking the effective working window so the agent can still summarize before it hits the wall. Overloaded is the worst case — everything maxed out, task completely invisible.
The connector on the right side shows how tool definitions produce results that land in conversation turns. Notice how the drain cascade works: when total content exceeds the effective window, headroom disappears first, then conversation gets compacted (reduced opacity bands), then context files shrink, and finally tool definitions yield — each slider shows the requested-vs-actual gap when drained.
The key insight: it's not about running out of tokens — it's about where your task lands relative to the attention curve. A 65% full window with heavy tool definitions is worse than an 80% full window where most content is conversation (which sits below the task in the recency zone).
This section describes behavior measured in long-context autoregressive LLMs, especially causal transformers. Diffusion language models denoise a sequence iteratively rather than generate left to right, so they do not share the same causal-mask and recency mechanism. Evaluate their positional reliability separately instead of assuming the same curve.
The Six Context Mechanisms
Context management
Six mechanisms share the same context lens
Each mechanism implements one or more of the four actions:
| Mechanism | Primary Action | Secondary Action |
|---|---|---|
| Context files (AGENTS.md) | Write | — |
| MCP tool schemas | Select | Compress (deferred loading) |
| Skills and procedures | Select | Write (when invoked) |
| Sub-agents | Isolate | Compress (synthesis return) |
| Context compaction | Compress | — |
| Retrieval (RAG, search, memory) | Select | Write (placement) |
The spectrum runs from always-loaded (context files — guaranteed delivery, high cost) to fully isolated (sub-agents — zero cost to parent, but no shared state). Effective context engineering means choosing the right mechanism for each piece of knowledge.
Context Files
Context files are markdown documents injected between the system prompt and your input — the primacy zone. They give agents project memory without repeated explanations: architecture, conventions, build commands, constraints.
The tradeoff: guaranteed delivery vs. lowering the task’s starting position. Every token in your context files makes the fixed prefix larger, so the user task enters lower in the window.
Context management
Always-loaded files lower the task’s starting position
AGENTS.md is the vendor-neutral standard adopted by 60,000+ open-source projects (now governed by the Linux Foundation's Agentic AI Foundation), working across GitHub Copilot, Cursor, Windsurf, and most other AI coding tools (note: Claude Code does not support AGENTS.md — see tip below). Keep it minimal — your README should contain 90% of what AI needs; AGENTS.md adds only AI-specific operational context. That 10% is where the discipline lives — every token in AGENTS.md consumes primacy-zone attention that could go toward your actual task. Put project knowledge in your README where agents can read it on demand; AGENTS.md is for what changes how the agent operates, not what the project does. Reference external docs by link rather than inlining them — agents can fetch details when needed.
Claude Code uses CLAUDE.md instead of AGENTS.md. To maintain a single source of truth while supporting both ecosystems, use @-linking in your CLAUDE.md:
# CLAUDE.md
@/AGENTS.md
This imports your AGENTS.md content into Claude's context. For details on Claude Code's hierarchical context system, see the Claude Code documentation.
Unlike AGENTS.md (a single file with predictable token cost), Claude Code's hierarchical system merges CLAUDE.md files from multiple levels — project root, subdirectories, .claude/rules/, user config, and enterprise policy — all automatically loaded based on your working directory. Each additional file consumes primacy-zone tokens, compounding the attention pressure this chapter describes: every merged file pushes your actual task deeper into the forgotten middle. The total context-file token cost becomes invisible unless you audit every level of the hierarchy. If you use Claude Code, keep a single root CLAUDE.md that @-imports AGENTS.md and avoid scattering files throughout subdirectories — the hierarchy multiplies the squeeze.
Context files are injected directly into system prompts. Security researchers have identified "Rules File Backdoor" attacks where malicious instructions are injected using Unicode characters or evasion techniques. Keep context files minimal, version-controlled, and code-reviewed like any other code.
MCP Tools
Every tool the agent can call ships with a schema: name, description, parameters. Those schemas live in the request context and count toward input tokens. Built-in tools (Read, Edit, Bash) are modest at ~50–1000 tokens each. MCP servers are a different story entirely.
The real issue is placement, not just capacity. Eager loading puts tool definitions in the initial request context before the model plans its first action. Deferred loading keeps only a search tool and compact catalog metadata upfront, preserving attention for the user task until a capability is actually needed.
The cost is structural: every tool schema is serialized as JSON with repeated type annotations, nested required arrays, and verbose natural-language descriptions. A typical tool runs 300–600 tokens; complex API tools like GitHub's file operations hit 1,000–2,000. These costs compound quickly across multi-server setups. The governing rule is simple: small, high-frequency toolsets favor eager loading; large, broad catalogs favor deferred loading.
Eager vs Deferred Loading
Eager loading keeps calls simple because the model already has the schema when it starts planning. Deferred loading protects the startup prompt by exposing ToolSearch first, but every deferred tool now depends on runtime discovery.
Context management
Load MCP schemas by catalog shape
Takeaway: the more your catalog behaves like infrastructure, the more you should optimize for prompt placement instead of immediate tool visibility. The same logic applies one layer up to reusable procedures: if a workflow library is broad, discover it cheaply and load the full instructions only when needed. The same automation-versus-reliability tradeoff applies here.
See the deferred-loading runtime sequence
When a deferred tool is needed, the model must discover → load → call:
SYSTEM
USER
AGENT
TOOL_RESULTAGENTDeferred loading is powerful precisely because it turns schema exposure into a runtime workflow instead of paying the full cost at startup.
Skills and Procedures
Skills are the procedure-side analogue of deferred MCP tools. Deferred MCP exposes a cheap capability catalog up front, then loads a tool schema only when the tool is needed. Skill systems do the same for workflows: the agent sees a lightweight listing first, then loads the full procedure only when the task matches or you invoke it directly.
The critical difference from tool schemas is not the mechanism — it is who decides which procedure to activate. This is the automation-vs-reliability tradeoff: auto-discovery is friction-free but the agent may pick wrong, while manual invocation always works but requires the human to know the skill exists.
Two Invocation Paths
Context management
Skills can be operator-triggered or model-invoked
Manual invocation: The user explicitly selects a procedure — a slash command (/commit), skill picker, or sub-agent mention. The procedure loads because the human chose it. Always reliable — the model cannot misidentify the workflow — but the human must discover the skill and remember its name.
Model invocation: The agent sees the procedure catalog in context, matches your intent against the descriptions, and loads the relevant workflow automatically. Zero friction — the agent handles discovery — but the model may pick the wrong skill, miss the right one, or waste tokens evaluating irrelevant descriptions.
Most skill systems allow per-skill configuration of this tradeoff. Claude Code exposes disable-model-invocation (manual only) and user-invokable (model only) as frontmatter fields in each SKILL.md. A skill with disable-model-invocation: true never auto-triggers — it runs only when explicitly invoked. This lets you match the invocation path to the workflow's reliability requirements: critical workflows (deploy, commit, review) benefit from manual invocation; exploration workflows (search code, explain errors) work well with auto-discovery.
Different Harnesses, Same Core Pattern
Unlike MCP — a defined protocol — skills are not standardized across tools. Each harness implements them differently. Yet they all follow the same pattern: advertise lightweight discovery metadata upfront, load the full procedure content only when invoked.
Claude Code lists skill names and descriptions, then loads SKILL.md when the skill is activated. Pi does the same and also exposes prompt templates as direct slash commands. Codex separates built-in slash commands from a /skills browser that inserts skill context only when selected. OpenCode leans more on reusable agents and @-invoked subagents, but the idea is identical: keep the startup prompt lean and bring in specialized instructions only when relevant.
That catalog listing is not free. Procedure metadata — names and descriptions used for discovery — consumes context budget even before any workflow runs. This is the procedure-side analogue of MCP schema pressure: different payload, same design problem.
The practical framing: AGENTS.md holds what the agent should always know — architecture, conventions, constraints. Skills hold what the agent should know how to do — specific workflows loaded on demand, with the invocation path controlling who decides which one runs.
Sub-Agents
Sub-agents run in completely separate context windows. This is the most powerful context engineering tool: a sub-agent might consume 50K–150K tokens researching a codebase, but returns only a 2K–5K synthesis to the orchestrator. Zero raw research tokens in the parent context — the search trail, file reads, and partial interpretations never existed as far as the main conversation is concerned.
Think of a sub-agent like a function call for agents. The prompt is the parameter: "Find all JWT authentication code and explain the current implementation." The synthesis is the return value: "JWT implementation found at src/auth/jwt.ts using Passport.js..." You pay for work in both contexts, but the parent receives the compact result instead of inheriting the whole research journey.
Context management
Sub-agents isolate context then return synthesis
The tradeoffs:
- No shared state — each sub-agent starts fresh, so you pay the context-file loading cost again
- Synthesis can lose nuance — the 2K summary omits details the parent might have needed
- Cost multiplied — every sub-agent is a separate API call with its own token consumption
- No nesting — sub-agents cannot spawn sub-agents; the orchestrator is always the root
- 10 parallel cap — Claude Code limits concurrent sub-agents to 10
The trade-off is token cost, not accuracy. You pay to process tokens in both contexts, but the orchestrator stays clean. That often improves first-iteration accuracy enough to save tokens compared with multiple correction cycles from a polluted parent context.
Sub-agents are ideal for heavy research, multi-file analysis, and exploratory tasks where the journey matters less than the destination. They can also generate independent plans or implementations when you want to compare options before applying one result.
Two Sub-Agent Architectures
Autonomous architecture: You give the agent tools and a system prompt. The agent autonomously decides what to do, which tools to call, and how to sequence its work.
Example: Claude Code's Explore agent. You send it a task, it autonomously picks tools and sequences, then delivers results. Dynamic and adaptable — it handles any task you throw at it without prior engineering.
Structured architecture: You build a deterministic control plane that defines the exact algorithm. The LLM makes tactical decisions within your structure ("Should I expand this symbol?" "Is this chunk relevant?"), but the overall flow is fixed.
Example: ChunkHound [disclosure] uses a fixed multi-hop pipeline where the system controls traversal strategy and the LLM ranks relevance at decision points. Deterministic and reproducible — same input, same results every time.
Context management
Two sub-agent architectures trade flexibility for reproducibility
The architectural trade-off: Autonomous agents are dynamic — they handle any task and adapt on the fly, but are non-deterministic (same prompt yields different paths each run) and bounded by context capacity. Structured agents are reproducible and scale to any information volume with better token efficiency, but only handle the specific task they were engineered for and cost more to build.
If You Cannot Use Sub-Agents
Without sub-agents, exploit the attention curve directly:
- Put durable constraints at the start, where primacy is strongest.
- Put the active task at the end, where recency is strongest.
- Keep supporting evidence in the middle summarized and referenced, not pasted wholesale.
- Start a fresh phase when the middle becomes crowded instead of continuing in a degraded thread.
This is less powerful than isolation, but it still respects the core rule: important facts must land where the model can attend to them.
Context Compaction
Compaction is lossy summarization. The model compresses the current session into a synthetic state guided by your instruction — but it cannot guarantee what survives. Future relevance is unknowable at compaction time: a debugging trace that looks like noise now may be the clue the next step needed.
The reliable alternative is manual handoff with external checkpointing. Before starting a new phase, write a brief state file with the current goal, changed files, decisions, unresolved errors, and next step. Files provide exact recall; a fresh context provides inference quality.
The Buffer Tax
Compaction itself consumes working context. The system reserves a buffer to ensure it can still perform the handoff when the session nears capacity — capacity intentionally withheld from normal conversation. This is the compaction buffer: a tax on your effective window.
Auto-compaction trades usable context for a safety margin you don't need when you own handoffs explicitly. Both Claude Code (CLI settings) and pi let you disable it:
{ "compaction": { "enabled": false } }
Reclaim the buffer for actual work. Compact only when you choose to — at phase boundaries — and always externalize anything that must survive exactly.
Retrieval
The agents chapter showed that memory is retrieval — the model does not remember your project conventions, previous sessions, or domain facts unless the harness retrieves them and feeds them into a later call. ChunkHound, OpenGrok, vector databases, BM25 indexes, web search, knowledge graphs, and memory systems are all retrieval flavors. They solve the same problem: choosing which facts deserve space in the next request.
Retrieval is a context engineering mechanism because it controls what enters the context window and where it lands. Two placement patterns dominate:
Harness-level RAG injects retrieved content before the user prompt — in the primacy zone. The agent sees it as background context. Memory systems, project indexes, and persistent knowledge work this way. The content gets primacy advantage but competes with context files for the fixed prefix.
Tool-call retrieval happens when the agent calls a search tool — ChunkHound, grep, web search — and the results land in the conversation stream, after the user prompt. This is how on-demand research works. The content lands in the recency zone initially, but drifts toward the middle as the conversation grows.
Each placement has different attention characteristics. Harness-level retrieval is always visible but always costs tokens. Tool-call retrieval loads on demand but competes with every other conversation turn for attention.
The Token Cost of Discovery
Retrieval decisions are context engineering decisions because every retrieved chunk costs tokens. Three grep searches can return 18,000 tokens. Reading five files can add another 22,000. You are at 40,000 tokens before discovery is finished. Ten semantic chunks from a vector database can cost 15,000 tokens; reading related files adds 25,000; exploring patterns adds another 10,000. You are at 50,000 tokens before implementation starts.
That is why correct retrieval can still hurt reliability. A long search trail can push the actual task into the forgotten middle. A useful web page can arrive too early, sit too long, and compete with more relevant instructions later. Coherent distractor text is especially dangerous because it looks relevant while degrading reasoning more than obvious noise.4
Retrieval Quality ≠ Context Quality
The failure mode most teams hit with retrieval is not "the retriever found irrelevant chunks." It is "the retriever found relevant chunks, but the model could not use them."5
Retrieval quality and context injection quality are separate problems. Finding the right facts is necessary but not sufficient. How those facts are structured, prioritized, and wrapped in the context determines whether the model can attend to them. A chunk that is relevant but arrives in the dead center of a 200K-token window may as well not exist.
This is why retrieval is a context engineering mechanism, not a precursor to it. The operator decides what to retrieve, where to place it, how much context budget to spend on it, and when to load it. Those are the same decisions that govern every other mechanism in this chapter.
Most providers apply per-token surcharges past a context threshold:
- Gemini 3.1 Pro: 2× input pricing above 200K tokens
- GPT-5.5 / GPT-5.4: 2× input pricing above ~272K tokens
- DeepSeek V4 Pro / Flash: 1.5× pricing above 512K tokens
- Claude 4.6/4.7 (Opus/Sonnet): No surcharge — flat rate to 1M
For retrieval tasks (summarization, document review), the surcharge buys tokens in the model's strong accuracy zone. For agentic coding, you're paying a premium for tokens in the model's worst accuracy zone — the middle, where your constraints and requirements go to die.
Key Takeaways
Write: Context files give guaranteed delivery at the cost of attention pressure Every token in AGENTS.md shifts the user task band deeper into the middle zone — keep files under 25KB. Put project knowledge in your README where agents can read it on demand; AGENTS.md is for what changes how the agent operates.
Select: Retrieval, MCP tools, and skills choose what enters the context Retrieval is the broadest select mechanism — ChunkHound, vector DBs, BM25, web search, memory systems all inject grounded facts. Harness-level RAG places content in the primacy zone; tool-call retrieval lands in the conversation stream. Retrieved content that is relevant but poorly placed still degrades. Optimize retrieval quality and context injection together.
A single GitHub MCP server consumes ~55K tokens; three moderate servers can consume 72% of your window before the conversation starts. Enable ToolSearch for deferred loading when you have more than 20 tools. Skills keep lightweight discovery metadata in context until invoked — the invocation path controls who decides which workflow runs.
Compress: Compaction and summarization fit more signal into fewer tokens Compaction is lossy summarization — it drops what looks irrelevant now but may be critical later. External checkpoint files provide reliable recall; a fresh context provides inference quality. If you must use compaction, externalize anything that must survive exactly.
Isolate: Sub-agents return synthesis without polluting the parent A sub-agent may consume 150K tokens internally but return only 2K to the parent — the research never touches the orchestrator's window. Use this for heavy research, multi-file analysis, exploratory tasks, and independent candidates that will be judged before one result is applied.
The U-curve is the constraint Position matters. Put critical instructions at the edges — primacy (start) and recency (end). Keep supporting evidence in the middle summarized and referenced, not pasted wholesale. Start a fresh phase when the middle becomes crowded.
Choose the mechanism by what the knowledge is for Is it always needed? → Write (context file). Is it a callable tool? → Select (MCP). Is it a workflow procedure? → Select (skill or prompt template). Is it heavy research? → Isolate (sub-agent). Is context accumulating? → Compress (checkpoint file and fresh phase). Is it domain knowledge or project state? → Select (retrieval).
Next: Chapter 7: Reliability Levers