Agentic search represents a fundamental shift from the traditional “retrieve-then-generate” paradigm to a dynamic, reasoning-driven approach where an AI agent actively controls the entire search process. Instead of firing a single query at a search index and hoping the results are sufficient, an agentic search system interprets user intent, decomposes complex questions into focused subqueries, evaluates retrieved results for quality and completeness, and iteratively refines its search strategy until it has comprehensive coverage.
The core insight is that most real-world questions are too complex for a single query. “How does our pricing compare to competitors for enterprise customers in the healthcare sector?” requires understanding pricing structures, identifying relevant competitors, and filtering for healthcare-specific terms — no single vector search can handle all of that simultaneously. Agentic search breaks this into focused retrieval operations and synthesizes the results.
Benchmarks from Azure AI Search show that agentic retrieval improves answer relevance by up to 40% compared to traditional RAG pipelines. This improvement comes from the agent’s ability to identify information gaps, generate targeted follow-up queries, and cross-reference results from multiple retrieval operations before producing a final answer.
At its core, agentic search follows an iterative cycle that extends the basic RAG pattern with reasoning and self-evaluation:
Traditional RAG: query → retrieve → generate (single pass, no evaluation). Agentic search: query → decompose → retrieve → evaluate → [retry if needed] → synthesize. The agent decides when to stop searching, not a fixed pipeline.
Query decomposition is the agent’s ability to break a compound question into atomic subqueries that each target a specific piece of information. An LLM analyzes the original question and generates focused subqueries based on the user’s question, chat history, and any constraints.
For example, the question “Is clickjacking different from SQL injection?” decomposes into:
Each subquery retrieves relevant context independently, and the results are combined for the final synthesis. This decomposition dramatically improves recall because each subquery is semantically focused rather than trying to match a broad, multi-faceted question.
// Query decomposition agent
async function decomposeQuery(userQuery: string, chatHistory: Message[]): Promise<string[]> {
const response = await llm.generate({
system: `You are a query decomposition agent. Break the user's
question into 2-5 focused subqueries that together cover all
aspects of the original question. Consider the chat history
for context. Return a JSON array of strings.`,
messages: [
...chatHistory,
{ role: "user", content: userQuery }
],
output: { schema: z.array(z.string()) }
});
return response.output;
}
// Execute subqueries in parallel
async function agenticSearch(userQuery: string, chatHistory: Message[]) {
const subqueries = await decomposeQuery(userQuery, chatHistory);
// Run all subqueries simultaneously
const results = await Promise.all(
subqueries.map(sq => hybridSearch(sq, { topK: 5 }))
);
// Flatten and deduplicate results
const allDocs = deduplicateByContent(results.flat());
// Evaluate completeness
const evaluation = await evaluateResults(userQuery, subqueries, allDocs);
if (!evaluation.isComplete) {
// Generate follow-up queries for gaps
const followUps = evaluation.gapQueries;
const additionalResults = await Promise.all(
followUps.map(sq => hybridSearch(sq, { topK: 3 }))
);
allDocs.push(...deduplicateByContent(additionalResults.flat()));
}
// Synthesize final answer from all retrieved context
return synthesizeAnswer(userQuery, allDocs);
}In recursive retrieval, the answer to each subquestion feeds into subsequent subquestions. This technique is essential for questions that require building context progressively — where the answer to “What is X?” informs how you search for “How does X affect Y?”
The recursive answering technique works as follows:
This ensures consistency across subquestions and prevents the agent from producing contradictory information, since each subsequent retrieval is informed by what was already established.
The evaluation step distinguishes agentic search from simple query expansion. After retrieval, the agent asks itself:
If the agent identifies gaps, it generates targeted follow-up queries. This loop continues until the agent determines it has sufficient information or reaches a maximum iteration limit (typically 3-5 rounds to prevent runaway costs).
Advanced agentic search systems route subqueries to different knowledge sources based on the type of information needed. The agent decides which source to query:
Recent research describes a shift from static, rule-driven retrieval pipelines to reasoning-driven architectures where the model embeds decision-making into the retrieval process itself. The agent actively determines when, what, and how to retrieve based on its internal reasoning trajectory — sometimes deciding that no retrieval is needed at all.
Corporate knowledge bases contain information scattered across wikis, Confluence pages, Slack threads, PDFs, and databases. A single-query RAG system struggles with questions like “What was our decision on the authentication migration and what were the security concerns raised?” Agentic search decomposes this into subqueries targeting the decision itself, the security review, and any follow-up actions — pulling from multiple sources and synthesizing a complete answer.
Market research questions are inherently multi-faceted. “How are competitors approaching AI-powered property tours in the hospitality space?” requires identifying competitors, finding their product announcements, analyzing their technical approaches, and comparing feature sets. An agentic search system can autonomously conduct this multi-step research, generating dozens of targeted queries across web search, industry databases, and internal competitive intelligence files.
Complex customer issues often span multiple product areas. A customer reporting “slow load times after the recent update on my iPad” requires searching across release notes, known bugs, device compatibility matrices, and performance benchmarks. Agentic search routes each aspect to the most relevant knowledge source and assembles a comprehensive troubleshooting response.
Legal questions require thorough coverage — missing a relevant regulation or precedent can have serious consequences. Agentic search’s iterative approach is particularly valuable here, as the agent can identify gaps in its initial retrieval and proactively search for additional relevant statutes, case law, or internal policies until it’s confident in comprehensive coverage.
Microsoft’s Azure AI Search implements agentic retrieval as a first-class feature. The system takes a user query and chat history, uses an LLM to generate focused subqueries, executes them in parallel with hybrid search (keyword + semantic), reranks results per subquery, and returns a unified result string. This is designed to be called from agents built on frameworks like LangGraph, Semantic Kernel, or Azure AI Foundry.