On-demand chat history retrieval is a memory architecture where an assistant starts each session without automatically injecting a large user profile, then calls explicit retrieval tools only when the current task needs prior conversations. The memory system behaves like a search interface over past chats rather than an always-on personalization layer.
This pattern is useful when fidelity, inspectability, and user control matter more than invisible personalization. Instead of compressing every prior interaction into a standing summary, the system keeps raw or lightly indexed conversation records outside the model context. The agent can issue a keyword search, a topic search, or a temporal query such as “recent chats” and then synthesize only the returned evidence into the current answer.
The source article describes this design through two visible retrieval tools: one for topic-based conversation search and one for chronological chat retrieval. The transferable engineering lesson is broader than any one product: persistent memory can be exposed as bounded, auditable tools with narrow schemas, result limits, project filters, and latency tradeoffs instead of being silently prepended to every prompt.
A blank-slate startup means the model does not receive a large memory block by default. The initial prompt contains the current request, system instructions, and active workspace context, but not an automatic dump of prior chats. This reduces prompt cost and avoids stale memories influencing unrelated tasks.
The agent accesses history by calling tools with explicit parameters. A topic search accepts a query string and a bounded result count. A recency tool accepts count, sort order, time cursors, or project filters. Tool calls create an observable boundary: traces show when memory was used, what query was issued, and which records came back.
The retrieval layer should return evidence from actual conversations: titles, timestamps, snippets, message excerpts, source identifiers, and links back to the original thread. The model then synthesizes those records for the user. This preserves provenance better than storing only model-generated memories, but it also requires careful result trimming so raw history does not flood the context window.
A search-backed assistant can recover relevant past context, but it has not internalized that context. It must decide when to search, phrase a good query, handle incomplete results, and explain uncertainty when the archive does not contain enough evidence.
The system first turns completed conversations into searchable records. Each chat keeps durable identifiers, timestamps, project scope, message metadata, and a searchable representation. The searchable representation may include a full-text index, embeddings for semantic search, and compact per-chat summaries used only for ranking.
CREATE TABLE chat_archive (
chat_id uuid PRIMARY KEY,
user_id uuid NOT NULL,
project_id uuid,
title text NOT NULL,
updated_at timestamptz NOT NULL,
messages jsonb NOT NULL,
search_text tsvector NOT NULL,
summary text,
embedding vector(1536),
deleted_at timestamptz
);
CREATE INDEX chat_archive_scope_idx
ON chat_archive (user_id, project_id, updated_at DESC)
WHERE deleted_at IS NULL;
CREATE INDEX chat_archive_text_idx
ON chat_archive USING gin (search_text);The agent receives small, typed tools rather than direct database access. A conversation search tool handles topic-oriented lookup. A recent chats tool handles chronological recall and pagination. Both enforce scope, maximum result counts, and output shaping before anything reaches the model.
{
"name": "conversation_search",
"description": "Search past conversations for relevant context.",
"parameters": {
"type": "object",
"properties": {
"query": { "type": "string" },
"max_results": {
"type": "integer",
"minimum": 1,
"maximum": 10,
"default": 5
},
"project_id": { "type": ["string", "null"] }
},
"required": ["query"]
}
}A production implementation usually combines lexical search, vector search, and temporal filtering. The retriever can run a hybrid query, rerank candidate chats by relevance and freshness, then return only bounded excerpts. The agent should receive enough evidence to answer, not the entire archive.
def conversation_search(user, query, max_results=5, project_id=None):
scope = authorize_memory_scope(user, project_id)
lexical = bm25_search(scope, query, limit=40)
semantic = vector_search(scope, embed(query), limit=40)
candidates = reciprocal_rank_fusion([lexical, semantic])
ranked = rerank_with_freshness(candidates, query)
return [
{
"chat_id": c.chat_id,
"title": c.title,
"updated_at": c.updated_at.isoformat(),
"excerpt": best_excerpt(c.messages, query, max_tokens=220),
"source_url": chat_url(c.chat_id),
}
for c in ranked[:max_results]
]When a user asks to continue work from a prior session, the agent can retrieve the most relevant project chats and summarize decisions, unresolved questions, and next actions. This avoids forcing users to paste old transcripts while keeping the current context compact.
Teams often need to know why an assistant claimed that a requirement, design decision, or preference exists. On-demand retrieval can include source links and timestamps, making memory influence reviewable in logs and user-visible citations.
Some users want continuity but do not want every conversation to shape every response. Tool-mediated memory lets the product expose controls such as incognito sessions, project-limited search, disabled history search, or per-chat exclusion without changing the core model.
The agent needs a policy for when memory lookup is appropriate. Obvious triggers include explicit phrases asking about prior conversations, continuation requests, and date-bounded questions. Ambiguous requests should usually ask a clarifying question or proceed without memory rather than searching unrelated private history.
def should_search_history(user_message):
explicit_patterns = [
"what did we discuss",
"continue where we left off",
"remember when",
"last time",
"our previous conversation",
]
text = user_message.lower()
return any(pattern in text for pattern in explicit_patterns)A single search endpoint tends to produce poor behavior for time-oriented prompts. Asking “what were my last few chats?” needs a chronological tool, not semantic similarity. Keep topic search and recent-chat retrieval separate so the model chooses the right access path.
The retriever should not over-summarize results into untraceable claims. Return structured evidence objects with excerpts, timestamps, and identifiers. Let the model synthesize and cite them, while the application preserves the raw provenance for debugging and export.