Manage the LLM Context as a Limited Engineering Resource

A coding agent performs best when its active context is small enough to remain in its "smart zone." As more material accumulates in the same context window, the model eventually enters a "dumb zone" in which its decisions become less reliable. The practical response is not simply to buy or consume a larger context window. It is to design tasks, sessions, and handoffs so that important reasoning happens early in a fresh context.

A new session begins with the system prompt and any instructions that are always loaded. The agent then usually moves through four broad stages:

1. It starts from the persistent system instructions.
2. It explores the codebase and gathers relevant information.
3. It implements a change.
4. It runs tests, type checks, or other feedback loops.

Every stage adds tokens. As the context grows, attention relationships become increasingly strained. A practical operating marker is to treat roughly 100,000 tokens as the boundary beyond which coding quality may begin to decline, even when the nominal context window is much larger. A million-token window may be useful for retrieving information from a very large body of text, but that does not mean the entire window is equally effective for difficult coding decisions.

The system prompt deserves special attention because it is present at the start of every session. A bloated system prompt can place the agent deep into the context budget before it has explored or implemented anything. Persistent instructions should therefore be kept as small as possible.

There are two basic ways to deal with a long session:

• Clear the context. This discards the conversation and returns the agent to the same known

starting state.

• Compact the context. This turns the session into a written history and continues from that

summary.

Compaction preserves continuity, but each compacted summary adds another layer of accumulated interpretation. Repeated compaction creates sediment: summaries of prior work become part of the next session, even when some details are no longer useful or accurate. A workflow that can repeatedly clear and restart from a stable base is easier to reason about because every new session begins from the same state.

Subagents provide another way to protect the main context. A subagent receives an isolated context window, performs a delegated exploration or analysis task, and returns only a concise summary to the parent agent. The delegated agent may consume a large number of tokens without filling the orchestrating agent's context with every intermediate detail.

Many apparent reasoning failures are actually context-management failures. An agent that performed well at the beginning of a session may start making poor architectural decisions, forgetting constraints, or applying inconsistent edits after a long exploration and implementation sequence. The problem is not always the model's raw capability. It may be that the most demanding work is being attempted after the context has already become overloaded.

Task sizing therefore becomes an engineering control. A large feature should be divided into pieces that can each be explored, implemented, tested, and reviewed within a fresh smart-zone session. This is the same discipline that helps human developers avoid becoming overwhelmed by oversized tasks.

Use the following operating pattern:

1. Display exact token usage during every coding session. Do not rely on a vague percentage

or intuition.

1. Keep persistent instructions small. Move optional guidance into pull-based skills or

task-specific prompts.

1. Delegate broad repository exploration to an isolated subagent when possible. Return only

the findings needed for the next decision.

1. Size each implementation issue so that one agent can understand it, make the change, and

run feedback loops before the context becomes degraded.

1. Clear the context between logically separate activities, especially between

implementation and review.

1. Prefer repeatable fresh starts over a chain of compactions when the workflow can

reconstruct the required state from code, issues, commits, and concise instructions.

Clearing a context removes useful conversational history. It works only when the workflow has externalized the important state into durable artifacts such as a product requirements document, issue files, code, tests, and commits.

Compaction is not useless. It can preserve a long-running discussion when restarting would be expensive. The limitation is that the compacted record becomes another interpretation layer, and repeated compaction can carry stale or distorted decisions forward.

A larger context window still has value for retrieval-heavy work. The mistake is treating advertised context capacity as an assurance that all tokens are equally useful for coding and reasoning.

Design the workflow so that exploration, implementation, and review occur in bounded, fresh contexts. Context size is not merely a model setting; it is a core engineering constraint that determines task size, session structure, and reliability.