Context Engineering Best Practices: Stop Feeding the Model Everything

The agent knew the rule. It had read the file. It had even followed the rule three turns earlier. Then the conversation grew, another tool returned a wall of JSON, and the same agent behaved as if the rule had never existed. The failure looked intelligent. The cause was much more ordinary.

Context engineering best practices treat an LLM window as a scarce attention budget. Write durable state outside the prompt, select only high-signal tokens for each turn, compress history before it rots, and isolate heavy work inside focused sub-contexts. Put durable rules first, rotate memory through the middle, and keep the live query plus fresh tool results near the end. Prefer just-in-time retrieval over full-file dumps, then measure context precision, recall, grounding, and tokens per task.

That is the direct answer. The harder lesson is that a larger window does not remove the need to design what enters it.

A reliable agent needs valves, not a bigger pipe. Write, select, compress, and isolate control what reaches the model's attention.

This context engineering guide covers the working system underneath reliable LLM applications. It will not turn every prompt into a perfect specification, and it will not pretend one database or agent framework solves the entire job.

What this guide covers

• Why prompt-only workflows fail as context grows

• The Write, Select, Compress, Isolate practice path

• Short-term and long-term AI context memory

• Schemas, tool definitions, and dynamic capability loading

• A vendor-neutral token budget and context pyramid

• Enterprise context platforms, including when to buy and when to build

• Evaluation through traces, precision, recall, grounding, and cost

• Nine practical context engineering FAQs

Why context engineering replaced prompt-only workflows

Imagine a theatre five minutes before opening night. The lead actor knows every line. The lighting board works. The props exist, somewhere, in twelve unlabelled crates. Yesterday's rehearsal notes sit beneath a takeaway menu, two people received different versions of the scene, and someone moved the door that matters in act three.

The actor can perform beautifully and still walk into a wall.

That is the useful way to understand context engineering for an LLM. The model is the actor. The prompt is one line of direction. The set, script version, prop placement, cues, permissions, rehearsal history, and current scene together form the environment in which that direction can succeed.

Prompt engineering improves the instruction. Context engineering designs the runtime information environment around it: system rules, retrieved facts, conversation state, tools, permissions, output formats, and the user's current request. LangChain's context engineering overview frames the discipline around what gets written, selected, compressed, and isolated. Anthropic's engineering guidance for agents makes the same practical point from another direction: success depends on curating the smallest set of high-signal tokens that a model needs next.

Stateless models and attention failure modes

An LLM is stateless between independent calls. Stateless means it does not carry a private memory of the previous request unless your application sends that information again or stores and retrieves it elsewhere. A chat interface can feel continuous while the machinery underneath repeatedly rebuilds the model's working environment.

So why not resend everything?

Because capacity and attention are different. A context window is the maximum amount of input the model can accept in one call. It is not a promise that every token inside that window will influence the answer equally. Research and practitioner testing in the source pack report a lost-in-the-middle effect, with accuracy dropping around 30 percent when important details sit in weak middle positions. The same ledger flags a functional correctness cliff around 32,000 tokens in cited Stanford and UC Berkeley work, even when a model advertises a larger theoretical limit.

The exact threshold will vary by model, task, and evaluation. The engineering lesson survives that variation: available space is not usable attention.

Long windows often favor information near the beginning and end. Critical instructions buried in the middle can become functionally invisible.

Rotary Position Embedding, usually shortened to RoPE, is a method transformers use to encode token positions. Across long sequences, positional effects can contribute to attention decay. You do not need to tune RoPE to act on the finding. You need to stop placing every rule, file, log, and chat turn into one undifferentiated block.

Contextual prompting is one useful layer

Contextual prompting adds relevant background to an instruction. You might include the user's role, the current code module, an example, or a constraint beside the request. That is useful, but it remains a prompt-level technique.

Context engineering owns the system that decides which background belongs there. It asks where that information came from, whether the user may access it, how fresh it is, where it should sit, when it should disappear, and how its effect will be measured. For a full treatment of that boundary, see context engineering vs prompt engineering.

Prompt engineering shapes an instruction. RAG retrieves information. Context engineering coordinates both with memory, tools, permissions, structure, and evaluation.

Context Engineering 2.0 and entropy reduction

The Context Engineering 2.0 framing from GAIR-NLP describes the job as entropy reduction. Here, entropy means uncertainty and disorder in the path from human intent to machine-executable context. A loose request arrives with missing assumptions, ambiguous entities, and scattered evidence. The context system turns that mess into a bounded task, approved sources, explicit tools, and a predictable output shape.

That sounds academic until you debug a production agent. Then it sounds like Tuesday.

The unreliable agent rarely needs another paragraph of encouragement. It needs fewer conflicting facts, a clearer boundary, and a way to preserve the decision it made two turns ago.

Write, select, compress, isolate: the practice path

The most practical framework in the research pack has four verbs: Write, Select, Compress, Isolate. They describe how information moves around an agent, not four prompt tricks.

The four practices form a loop. State leaves the window, returns when relevant, shrinks as it ages, and moves into separate contexts when work becomes noisy.

1. Write durable state outside the window

Writing means moving useful information from transient conversation into persistent storage. That storage might be a task note, database row, semantic cache, user profile, repository file, or structured memory object. The format matters less than the contract: preserve the fact, its source, its timestamp, and the scope in which it remains valid.

Do not save every utterance. Save decisions, constraints, unresolved questions, identifiers, and outcomes. A raw chat transcript is an audit artifact, not automatically good memory.

For example, an agent working on authentication should persist the chosen session strategy and the reason for it. It should not reload twenty pages of exploratory discussion every time it edits a middleware function.

2. Select with just-in-time retrieval

Selection decides what returns. Just-in-time retrieval means fetching information when the current step needs it instead of preloading the entire knowledge base. Keep lightweight handles such as file paths, record IDs, entity keys, and search terms in the working context. Resolve them into full content only when the task crosses that boundary.

Retrieval-Augmented Generation, or RAG, retrieves external material to augment a model request. Hybrid RAG combines semantic similarity with exact filters such as date, entity, repository path, or permission. This tends to beat a pure similarity search when a nearby but outdated document would be dangerous.

A simplified retrieval tool from the LlamaIndex pattern looks like this:

def search_knowledge(query: str) -> str:
    """Retrieve knowledge relevant to the current subtask."""
    nodes = retriever.retrieve(query)
    return "\n----\n".join(node.text for node in nodes)

Production code still needs freshness filters, authorization, ranking, and provenance. The point is the interface: the model asks for a narrow slice when it needs one.

Selection is an editorial act. An authentication task needs the relevant routes, middleware, schema, and tests, not a ceremonial dump of the whole repository.

3. Compress history and clear tool noise

Compression turns a long trajectory into a smaller state that preserves decisions and open work. Anthropic's source material describes compaction and tool-result clearing as ways to keep long-running agents useful. The claim ledger records an auto-compaction pattern near 95 percent of window capacity. Treat that as a sourced implementation pattern, not a universal setting for every model or product.

A good compacted state contains:

• The current objective and acceptance criteria

• Decisions made, with their reasons

• Files, entities, or records changed

• Evidence already checked

• Failed approaches that should not repeat

• Remaining tasks and known blockers

Tool-result clearing handles a more specific problem. Once a tool's raw output has served its purpose, replace the payload with the small fact or reference the agent still needs. Ten kilobytes of search JSON should not follow the agent through another fifteen turns because one field mattered.

For implementation-specific memory and compaction patterns, continue with effective context engineering for AI agents.

4. Isolate work without multiplying confusion

Isolation gives a noisy subtask its own context. A planning agent can read architecture and requirements. A coding agent can receive the plan, relevant modules, and local rules. A testing agent can inspect the implementation plus test conventions. Each gets a focused budget instead of inheriting the complete history of every other worker.

Isolation protects attention, but each handoff needs an explicit schema. Otherwise, separate windows produce separate assumptions.

There is a real trade-off here. Anthropic reports a 90.2 percent improvement for a multi-agent research system on its cited long-horizon evaluation, alongside the possibility of roughly 15 times the token usage. The synthesis also records the competing Cognition-style view: a single thread with deliberate boundary compaction can be cheaper and more stable because parallel agents make uncoordinated implicit decisions.

Both can be true. Use isolation when subtasks can produce explicit artifacts, require different evidence, or would flood one window. Prefer a single thread when decisions are tightly coupled and coordination would cost more than the work.

This will not work if the handoff says only "research this" or "implement the feature." Define the task, allowed sources, constraints, output schema, and definition of done. Isolation without a contract is just amnesia with extra invoices.

5. Keep repository rules specific

Files such as CLAUDE.md, AGENTS.md, and .cursorrules can carry persistent repository guidance. Put stable, codebase-specific information there: commands that actually run, architectural boundaries, directories that must not change, test requirements, and one or two representative examples.

Avoid generic filler such as "write clean code" or "follow DRY." DRY means Don't Repeat Yourself, a software design principle about avoiding duplicated knowledge. It becomes useful only when you identify what duplication means in this repository and where the shared abstraction belongs. Faros reports a roughly 35 to 40 percent reduction in style violations when rules become specific, but that is a vendor tutorial metric, not a universal benchmark.

AI context memory: short-term, long-term, and GraphRAG

AI context memory is not one giant transcript. It is a set of stores with different lifetimes and retrieval rules.

Short-term memory holds the active session: recent turns, current tool results, local task state, and a working summary. Long-term memory holds durable information across sessions: preferences, prior decisions, verified facts, entity histories, and institutional knowledge. A consolidation process promotes selected facts from the first layer into the second.

Memory becomes useful when the system consolidates selected facts with provenance, rather than treating every conversation line as equally durable.

The simplest design test is deletion. If removing an item would break the current turn, it belongs in working memory. If removing it would make the system repeat a settled mistake next month, it may belong in durable memory. If neither happens, you probably do not need to store it.

When graphs beat flat vector retrieval

Vector embeddings represent meaning as numeric coordinates, which makes them good at finding semantically similar text. A knowledge graph stores entities and explicit relationships such as customer owns account, service depends on database, or policy supersedes policy. GraphRAG combines graph traversal with retrieval for questions that require connected facts.

Use vectors when topical similarity is the main job. Use a graph when answers depend on multi-hop relationships, duplicate entity resolution, temporal changes, or contradictory claims from different systems. Use a hybrid when users ask both kinds of questions.

Choose memory architecture by reasoning pattern. Similarity favors vectors; connected and changing facts favor graphs; mixed workloads often need both.

Format optimization and structured schemas

Formatting is not decoration. It gives both sides of the interaction a contract.

On the input side, delimiters separate instructions, retrieved evidence, conversation state, and the live query. XML tags and Markdown headings both work when they remain consistent. The DAIR.AI context engineering guide gives patterns such as wrapping the current request explicitly:

<user_query>
What's the latest dev news from OpenAI?
</user_query>

The tags do not make the model smarter. They make boundaries legible. That reduces the chance that quoted data looks like an instruction or that an instruction disappears inside retrieved text.

On the output side, JSON or YAML schemas force the agent to return fields downstream code can validate. Bilateral context means the system structures both what the model receives and what it must return.

task: "Check authentication middleware"
query: "Find current session validation rules"
constraints:
  - "Use authorized repository sources only"
  - "Return source paths for every claim"
expected_format: "findings_with_evidence"

A schema turns a vague delegation into a testable handoff. Required fields expose missing context before downstream tools inherit it.

Do not overbuild the schema. Every field consumes tokens and creates another failure condition. Include what the next step will validate or use.

Tools, function specs, and MCP loadouts

Every active tool brings a description, parameters, examples, and potential ambiguity into the model's environment. Give an agent thirty near-duplicate tools and it must spend attention choosing among them before it can do the user's work.

Start with a minimal viable toolset. Each tool should have one distinct job, a precise description, typed parameters, and a return shape that does not require guesswork. CamelCase names can tokenize more efficiently than punctuation-heavy names, according to the Comet guidance in the source pack, but naming cannot rescue overlapping functions.

A smaller, distinct loadout reduces tool confusion. Dynamic routing can fetch extra specifications only when the task calls for them.

For a large catalog, retrieve tool descriptions just in time. This is RAG over capabilities: a router selects the few tool specifications relevant to the query rather than exposing every integration on every turn.

The Model Context Protocol, or MCP, is a standard way for AI applications to connect models with tools and data sources. Treat MCP servers as a dynamic capability surface, not permission to load an entire company's tool inventory into every request. Authorization still belongs upstream. Selection still belongs in the context pipeline. For framework-level routing patterns, see LangChain context engineering.

Vendor-neutral token budgeting and the context pyramid

Most vendor guides tell you to retrieve better. Fewer tell you where the retrieved material should sit or what it must displace.

A token is a unit of text processed by the model. A token budget is the amount of the window you deliberately allocate to rules, tools, memory, evidence, live work, and response space. This is not merely a hard maximum. It is an attention plan.

The quiet operational truth is that a token earns its place only when removing it makes the task worse.

Functional cliffs matter more than marketed capacity

Begin with the model's advertised context capacity, then set a lower operational ceiling based on evals. The source ledger's 32,000-token correctness cliff and approximately 30 percent middle-zone drop are warnings, not constants you should paste into configuration. Your own task distribution decides the safe ceiling.

Track four zones:

1. The front holds durable constraints, security rules, and the minimum tool definitions.

2. The early-middle holds repository structure, selected examples, and rotating durable memory.

3. The late-middle holds current state, recent outcomes, and compressed history.

4. The working edge holds the exact query, active files or evidence, and fresh tool results.

Always reserve output capacity. A request that fills the complete window leaves the model no room to reason or answer.

Build a persistent, dynamic, and ephemeral pyramid

Neo4j's source pattern groups the same idea into three layers. The Persistent Base contains system rules, safety requirements, tool contracts, and cached facts that apply broadly. The Dynamic Layer rotates examples, memory, repository context, and metadata per task. The Ephemeral Layer contains the current query and raw tool outputs, which should disappear or compress quickly.

Persistent rules form the base, dynamic memory rotates with the task, and ephemeral query data stays near the working edge.

Here is a practical layout procedure for one agent window:

1. Write the task's acceptance test in one sentence. If you cannot, stop and clarify the specification.

2. List the rules whose absence could make the output unsafe or invalid. Put those near the front.

3. Load only the tool contracts the current step can call.

4. Retrieve dynamic memory and evidence against the task, permissions, freshness, and entity filters.

5. Compress old dialogue into decisions, failures, and open work.

6. Place the live request and fresh results near the working edge.

7. Reserve answer space, run the task, and record which context items the model actually used.

8. Clear raw results and write durable outcomes back to storage.

The worksheet makes displacement visible. Adding a file is a budget decision because something else loses attention or leaves the window.

You can now test density instead of admiring capacity. Remove one context block and rerun the eval. If quality holds, the block was cost. If quality falls, inspect whether the block was necessary evidence or merely compensating for a weak task specification.

Enterprise context platforms: evaluate vs build

Application context engineering and enterprise context management overlap, but they are not the same purchase.

An application team can build a focused pipeline around one agent: a session store, retrieval service, a handful of tools, and a trace collector. An enterprise platform must solve cross-team concerns such as lineage, role-based access control, entity resolution, freshness contracts, data discovery, and real-time delivery. Lineage records where data came from and how it changed. Role-based access control, or RBAC, limits data and actions according to a user's approved role.

The right question is not which context engineering products enterprise teams are talking about. It is which shared problem has become expensive enough to centralize.

Build context backwards from governance

Retrieval cannot fix a source system that mixes duplicate customers, stale policies, and unauthorized records. Resolve entities before indexing. Attach provenance and update timestamps. Enforce permissions before retrieval results reach the model. Then design the prompt-facing representation.

This is where many context engineering AI products for enterprise use cases start from the wrong end. They optimize similarity search, then discover that the most similar document belonged to another region, another customer, or last year's policy.

Index-first systems inherit stale data, missing lineage, over-retrieval, and access failures. Context reliability begins before the vector search.

Build a narrow layer when one team owns the sources, access model, and latency target. Evaluate a platform when multiple applications need the same governed memory, catalog, entity graph, or retrieval contracts. Pay particular attention to five patterns: a low-latency memory layer, governed data products, GraphRAG, an analytical store for joining state with operational data, and a metadata catalog for lineage and discovery.

Evaluate platforms by the shared job they remove. Product categories are useful; a vendor logo parade is not.

Centralization also has a cost. A platform can slow local experiments, flatten domain-specific ranking, and create a new dependency for every request. Keep the application boundary clear: the platform supplies governed context capabilities, while the application still decides what this task needs now. For the platform category itself, see agentic context engineering platforms.

Evaluation, tracing, and observability

You cannot improve a context pipeline by reading final answers alone. You need traces.

A trace records the steps of a run: input context, retrieved chunks, tool calls, intermediate decisions, output, latency, and token use. Tools such as LangSmith and Opik can capture these events, but the metrics should remain portable.

Track at least four:

• Context precision: Of the context supplied, how much was relevant or used?

• Context recall: Of the evidence needed for a correct answer, how much did retrieval supply?

• Evidence grounding: Do important output claims connect to approved evidence?

• Tokens per task: How many input and output tokens did a successful run consume?

Precision catches noise. Recall catches missing facts. Grounding catches unsupported confidence. Tokens per task catches expensive architectures that look clever in demos.

Add slice-based evals. Compare short and long sessions, fresh and stale sources, authorized and unauthorized users, simple and multi-hop questions, single-agent and isolated runs. An average score can hide the exact failure your customers will find first.

I tried judging one pipeline by answer quality alone, and it looked excellent. The trace showed that the agent had retrieved the right policy, ignored it, and answered from a stale conversation summary. The output passed by accident. That is why observability belongs inside context engineering, not after it.

Frequently asked questions

How to make an AI model become context aware?

You make a stateless model context-aware by building a runtime system around it. Store durable rules and memory outside the model, retrieve relevant facts and user state for each turn, inject approved tool outputs, and structure the result before generation. The model itself does not magically remember; your application reconstructs the right working environment.

Why isn't prompt engineering enough anymore?

Prompt engineering cannot supply missing data, repair stale memory, enforce upstream permissions, or prevent a large window from filling with noise. It remains valuable for clear instructions, but reliable systems also need retrieval, state, tools, compaction, access control, and evaluation. The deeper comparison belongs in context engineering vs prompt engineering.

What is the difference between context engineering and RAG?

RAG is a retrieval mechanism that fetches external information for a model request. Context engineering governs the complete runtime environment, including instructions, RAG results, memory, tool definitions, permissions, ordering, compression, and output schemas. RAG can be one component of a context system.

What is context engineering in simple terms?

Context engineering is the practice of giving an LLM the right information, tools, rules, and memory for the current task, in a form and order it can use. It replaces the habit of dumping everything into a prompt with a system that selects and maintains context deliberately.

Can good context engineering compensate for a bad task specification?

No. Context engineering defines the environment and available evidence, while the task specification defines what success means. A perfectly curated window can still produce the wrong product if the acceptance criteria are vague or contradictory. Clarify the job before optimizing the context.

How do you keep context from drifting over time?

Compact long histories into decisions and open work, clear raw tool outputs after use, preserve durable facts with provenance, and enforce freshness rules on retrieval. Then run regression evals across long conversations. Drift becomes manageable when the system can distinguish current state from historical debris.

When should I use a knowledge graph instead of vector embeddings?

Use a knowledge graph when the answer depends on relationships, multi-hop paths, duplicate entities, temporal changes, or contradictory facts. Use vector embeddings when semantic similarity is enough. A hybrid approach fits systems that need both topical retrieval and explicit relationship reasoning.

How does context pruning reduce LLM costs?

Providers meter model usage by tokens, so removing irrelevant history, duplicated instructions, verbose JSON, and stale retrieval results reduces input volume. Good pruning can also improve quality by reducing attention noise. Measure successful tokens per task rather than assuming shorter is always better.

Why do large context windows still lose information?

Models do not attend equally to every position in a long sequence. Important details can weaken in middle zones, while early instructions and recent content receive stronger influence. A large advertised capacity therefore increases available space without guaranteeing reliable recall across that entire space.

Try this on one failing agent loop

Pick one workflow that has failed twice in the same irritating way. Do not rebuild its stack. Draw three boxes labeled Persistent, Dynamic, and Ephemeral. Place every current input into one box, then circle anything whose removal would actually change the result.

Apply one intervention. Write a decision outside the chat. Select one file instead of a repository. Compress a completed tool call. Isolate a research task behind a structured handoff. Add one metric to the trace and compare the next ten runs with the previous ten.

If several applications need the same memory, permissions, lineage, or entity resolution, evaluate a shared platform. If one agent needs three cleaner files and a better summary, keep the solution local. Architecture should grow when the shared problem grows, not when the diagram looks lonely.

The next generation of agent systems will probably advertise even larger windows. Useful. But the builders who win will still know what deserves to enter, what needs to leave, and what must never cross the boundary in the first place.

Until then...

• Sage

PS. Take the longest prompt in your system and remove one block you are certain is essential. Run the eval before reading the diff. Your confidence and the trace may disagree, which is exactly the lesson.