Vectorless tree-search RAG replaces the usual chunk-embed-search loop with an explicit document navigation process. Instead of splitting a long document into fixed-size chunks and ranking them by embedding similarity, the system builds a hierarchical index from the document structure and asks a language model to reason over that tree. The retriever behaves more like a human analyst using a table of contents: choose a likely section, inspect it, decide whether the evidence is sufficient, and continue navigating if needed.
PageIndex is a concrete implementation of this pattern. Its README describes a two-step retrieval process: generate a table-of-contents-like tree structure, then perform reasoning-based retrieval through tree search. The approach is designed for long professional documents such as financial reports, legal filings, technical manuals, regulations, and textbooks where semantic similarity often finds text that sounds related but is not the right evidence.
The technique is not a universal replacement for vector search. It is most valuable when each document has meaningful internal structure, the answer depends on locating the right section rather than the most similar phrase, and traceability matters. For broad multi-document recall, vector or hybrid search may still be the better first-stage retriever.
Dense retrieval assumes that the most semantically similar chunk is likely to be the most useful context. That assumption breaks when many sections share the same vocabulary but answer different questions. Financial reports, contracts, and manuals often repeat terms across sections, while the relevant answer depends on reporting scope, definitions, appendices, or exceptions.
The tree index is a compact structured representation of the document. Each node has an identifier, title, summary or description, page or section bounds, and children. The model can inspect this structure directly, choose nodes, and retrieve raw content by node id.
The retriever performs an iterative loop: read the index, choose a promising node, fetch that node’s content, decide whether the content is sufficient, and either answer or continue with neighboring or child nodes. The retrieval trace is a path through the tree rather than an opaque vector-nearest-neighbor result.
The key shift is that retrieval becomes an agentic decision process. The model does not merely score chunks; it reasons about document structure and intentionally chooses where to look next.
The indexing pass extracts a logical tree from the document. For PDFs this may require layout-aware parsing or vision-based extraction; for Markdown, heading levels can define the tree directly. Each node should point back to retrievable raw content.
{
"node_id": "0006",
"title": "Financial Stability",
"start_index": 21,
"end_index": 22,
"summary": "Section summary for retrieval planning",
"nodes": [
{
"node_id": "0007",
"title": "Monitoring Financial Vulnerabilities",
"start_index": 22,
"end_index": 28,
"summary": "Child section summary"
}
]
}The query-time loop keeps the tree or a compressed portion of it in context. The model selects nodes, calls a retrieval function for raw node content, and evaluates whether the evidence answers the question.
def tree_search_retrieve(question, index_tree, fetch_node, max_steps=6):
evidence = []
cursor = index_tree
for _ in range(max_steps):
choice = llm_choose_node(question, cursor, evidence)
node_text = fetch_node(choice.node_id)
evidence.append((choice.node_id, node_text))
if llm_evidence_is_sufficient(question, evidence):
return evidence
cursor = choice.subtree_or_sibling_context()
return evidenceThe answer generator receives only the selected node contents and their source identifiers. This produces more inspectable citations than a raw similarity list because the path explains why the system moved through the document hierarchy.
Annual reports and filings contain repeated metrics, definitions, tables, and footnotes. Tree-search retrieval can first navigate to the relevant statement, then inspect footnotes or appendices when the answer depends on a definition or adjustment.
Contracts and regulations are highly structured. A model can navigate clauses, definitions, exceptions, and referenced appendices instead of relying on keyword overlap between a query and a chunk.
Large manuals often contain cross-references such as “see Appendix G” or “refer to Table 5.3.” A tree index gives the retriever a way to follow those references intentionally.
When OCR destroys layout, a vision model or layout parser can produce a structure-aware index over page images. Retrieval can then preserve section and page context rather than flattening the document into arbitrary text chunks.
Use vector search to find candidate documents when the corpus is large, then use tree-search retrieval inside the winning document. This hybrid pattern keeps vector search where it is strong while avoiding chunk confusion inside long structured files.
candidate_docs = vector_search_corpus(query, top_k=5)
ranked_doc = rerank_documents(query, candidate_docs)[0]
evidence = tree_search_retrieve(
question=query,
index_tree=load_tree(ranked_doc.id),
fetch_node=lambda node_id: load_node_text(ranked_doc.id, node_id),
)
answer = generate_grounded_answer(query, evidence)Node summaries should help the retriever decide where to inspect, not replace the source text. Summaries that are too answer-like can hide important exceptions; summaries that are too generic make node selection unreliable.
The navigation loop is naturally tool-driven. Provide tools such as list_children(node_id), fetch_node(node_id), fetch_neighbor(node_id), and search_titles(query). Keep the tool contract small so the model’s retrieval trace remains auditable.