Context engineering is the architectural discipline of curating the entire set of tokens that populate an LLM’s context window. While prompt engineering focuses on how you phrase a request, context engineering addresses the broader question: what information does the model need access to right now, and in what structure, to accomplish the task? It treats the context window as a finite, precious resource — an “attention budget” — that must be allocated strategically across system instructions, retrieved data, conversation history, tool definitions, and user input.
The shift from prompt engineering to context engineering reflects the evolution of LLM applications from simple single-turn interactions to complex, multi-turn agentic systems. In single-turn use cases, a well-crafted prompt suffices. But in production AI systems where an agent reasons across multiple steps, calls tools, retrieves external knowledge, and maintains conversation state, the entire context window becomes an engineered artifact that must be actively managed.
As Andrej Karpathy put it, context engineering is about “building dynamic systems that provide the right information and tools, in the right format, such that the LLM can plausibly accomplish the task.” Gartner went further in mid-2025, declaring “context engineering is in, and prompt engineering is out” — signaling that this discipline is now a core competency for AI engineers building production systems.
A well-engineered context window is built from six interdependent components that control what information reaches the model and when:
LLMs have a finite attention budget. As the context window fills, the model’s ability to accurately recall and reason over earlier content degrades — a phenomenon called context rot or catastrophic context collapse. Context engineering is fundamentally about maximizing signal-to-noise ratio within this budget: finding the smallest possible set of high-signal tokens that maximize the likelihood of a desired outcome.
The distinction is hierarchical. Prompt engineering is a subset of context engineering, not the other way around:
A perfectly worded prompt will fail if the wrong data is in the context window. Conversely, the right data with a mediocre prompt often succeeds. This is why production AI systems invest heavily in retrieval pipelines, memory management, and token budgeting — the context assembly logic is often more complex than the prompts themselves.
Consider a typical 128K-token context window for a production RAG agent. A context engineer might allocate the budget like this:
System prompt & persona: ~2,000 tokens ( 1.5%)
Tool definitions (8 tools): ~3,000 tokens ( 2.3%)
Retrieved context chunks: ~20,000 tokens (15.6%)
Conversation history: ~10,000 tokens ( 7.8%)
Current user query: ~500 tokens ( 0.4%)
Reserved for generation: ~4,000 tokens ( 3.1%)
─────────────────────────────────────────────────────
Allocated: ~39,500 tokens (30.9%)
Remaining buffer: ~88,500 tokens (69.1%)The key insight is that even with large context windows, you should not fill them indiscriminately. Research consistently shows that model performance degrades when context windows are packed beyond ~30-40% with heterogeneous content. The buffer exists for multi-turn expansion and agentic tool-use loops.
In agentic systems, context is not static — it changes on every inference call. A context engineering pipeline dynamically assembles the context based on the current state:
function assembleContext(userQuery, conversationHistory, agentState) {
// 1. Query augmentation
const enrichedQuery = augmentQuery(userQuery, conversationHistory);
// 2. Retrieve relevant chunks
const chunks = await hybridSearch(enrichedQuery, { topK: 10 });
// 3. Rerank and prune to budget
const reranked = await rerank(enrichedQuery, chunks);
const pruned = fitToBudget(reranked, TOKEN_BUDGET.retrieval);
// 4. Select relevant tools
const tools = selectTools(enrichedQuery, agentState);
// 5. Compress conversation history
const history = compressHistory(conversationHistory, TOKEN_BUDGET.history);
// 6. Assemble final context
return {
systemPrompt: SYSTEM_PROMPT,
tools: tools,
messages: [...history, ...pruned, { role: 'user', content: userQuery }]
};
}Each component — retrieval, tool selection, history compression — is a separate engineering concern that must be tuned independently. The context assembler orchestrates them into a coherent whole that fits within the token budget.
For multi-turn conversations and agentic loops, context engineering becomes stateful. Key strategies include:
In a production RAG pipeline, context engineering determines how many retrieved chunks to inject, how to order them (most relevant first, or most relevant last to exploit recency bias), whether to include metadata like source URLs and timestamps, and how to format chunks so the model can distinguish between different sources. Getting this wrong leads to hallucinations, missed information, or context rot.
AI agents that execute multi-step plans face an escalating context problem. Each step adds tool calls, observations, and intermediate reasoning to the context. Without context engineering, the agent “forgets” its original objective after 5-10 steps. Strategies like summarizing completed subtasks, pruning tool call details after execution, and maintaining a separate “scratchpad” for the current subtask are all context engineering techniques.
A customer support AI must juggle the user’s current question, their account history, relevant knowledge base articles, company policies, and persona instructions — all within a single context window. Context engineering here involves real-time decisions about which customer data to include (recent tickets? purchase history? account status?) and aggressive pruning of irrelevant knowledge base results.
In systems like Visiting Media’s AI-powered property search, context engineering determines how property metadata, guest preferences, conversation history, and search results are assembled into a context that produces accurate, personalized responses. The token budget must accommodate structured property data alongside natural language conversation — a classic mixed-modality context engineering challenge.