Relevance feedback is a retrieval pattern where the system uses judgments about the first result set to improve the next result set. Classic full-text search systems used it for query expansion and Rocchio-style vector updates. Modern vector search systems need a version that works directly over dense embedding indexes, because agentic workloads often issue many retrieval passes and cannot afford slow query-rewrite loops, large reranking batches, or per-request model fine-tuning.
Vector-native relevance feedback keeps the fast retriever in the loop. The first search returns a small context window. A feedback model, user action, or lightweight classifier scores those few results for relevance. The vector engine then converts the feedback into a modified similarity function for the next retrieval pass, so the index traversal itself is nudged toward candidates that look more like the positive examples and less like the negative examples.
The important distinction is where the feedback is applied. A reranker only reorders candidates already found by the first retriever. A vector-native feedback query changes how candidates are scored during the next search over the full vector space. That makes it useful when the right documents are not merely misordered, but absent from the first top-k set because the base embedding model is too weak, too generic, or misaligned with the task.
The feedback signal can come from many sources: explicit user likes and dislikes, click logs, model-generated scores, cross-encoder judgments, a stronger embedding model, or an LLM that compares the query to a few returned snippets. The feedback does not need to label the entire corpus. It only needs to describe which examples in the initial context are more or less relevant to the current query.
A context pair is a positive and negative example extracted from the first result set. If the feedback model says one result is clearly closer to intent than another, that pair defines a direction in embedding space. The next retrieval pass can reward candidates closer to the positive example and penalize candidates closer to the negative example while still preserving similarity to the original query vector.
Most retrieval pipelines use feedback after search: fetch candidates, score them, sort them, and return the top results. Vector-native relevance feedback pushes the adjusted score into the search operator itself. The vector engine still traverses an ANN index, but the candidate score becomes a formula that blends query similarity with feedback-derived terms.
Vector-native feedback is not just a second-stage ranking feature. It alters the distance or similarity signal used by the next retrieval pass, allowing the engine to surface candidates the original query vector did not rank highly enough.
The pipeline begins with ordinary semantic retrieval. The base retriever embeds the query and asks the vector index for a small number of candidates. The feedback model then scores those candidates. For latency control, the context should be small: a few documents are enough to create useful positive-negative pairs, while hundreds of documents would turn feedback into an expensive reranking workload.
from qdrant_client import QdrantClient, models
client = QdrantClient(url=QDRANT_URL, api_key=QDRANT_API_KEY)
query = "recommendations API how to use"
query_embedding = retriever.embed_query(query)
context = client.query_points(
collection_name="documentation",
query=query_embedding,
with_payload=True,
with_vectors=True,
limit=3,
).points
texts = [point.payload["text"] for point in context]
feedback_scores = feedback_model.score(query, texts)The search engine needs feedback in a form it can attach to indexed points or raw vectors. One implementation accepts a target query vector plus feedback items, where each item contains an example point or vector and a relevance score. Higher-scored examples become positive anchors; lower-scored examples become negative anchors. The feedback strategy then converts those anchors into parameters for an adjusted scoring formula.
point_ids = [point.id for point in context]
feedback_items = [
models.FeedbackItem(example=point_id, score=score)
for point_id, score in zip(point_ids, feedback_scores)
]The second pass runs a relevance feedback query instead of a plain nearest-neighbor query. Conceptually, the score for each candidate combines similarity to the target query, closeness to positive examples, distance from negative examples, and strategy parameters learned or tuned for the collection, retriever, and feedback model. The result is still a vector search, but the search direction has been informed by the first result set.
formula_params = {"a": 0.24, "b": 1.35, "c": 0.59}
results = client.query_points(
collection_name="documentation",
query=models.RelevanceFeedbackQuery(
relevance_feedback=models.RelevanceFeedbackInput(
target=query_embedding,
feedback=feedback_items,
strategy=models.NaiveFeedbackStrategy(
naive=models.NaiveFeedbackStrategyParams(**formula_params)
),
)
),
with_payload=True,
limit=10,
).pointsAgents often retrieve repeatedly: expand the query, inspect results, gather missing context, and run another search. If every loop uses only text query rewriting, the agent pays extra model latency and may drift away from the original intent. Relevance feedback lets the agent score a tiny context window, then ask the vector engine to search in a better direction without generating a large rewritten query set.
Many production systems use small embedding models because they are cheap, fast, and easy to serve. A small retriever may miss relevant documents for subtle queries. A stronger but more expensive feedback model can inspect only the first few results and steer the second pass, improving recall without replacing the entire index or embedding every query-document pair with a cross-encoder.
End users can provide explicit feedback by marking results useful or irrelevant. Product search, legal discovery, documentation search, and image search can all use those clicks or labels to run a refined retrieval pass. The mechanism works beyond text because the adjusted scoring operates on vectors and example IDs, not only on rewritten natural-language queries.
A feedback strategy usually has tunable parameters that control how strongly positives, negatives, and the original query affect scoring. Tune those parameters on realistic queries before enabling the path in production. The tuning set can be modest because only a few formula weights may be learned, but it should reflect the actual collection and query distribution.
train_queries = load_realistic_queries()
formula_params = relevance_feedback.train(
queries=train_queries,
limit=25,
)
if formula_params.validation_gain < MIN_GAIN:
disable_relevance_feedback()The first-pass context window is a budget knob. Larger windows give the feedback model more evidence, but they increase latency and cost. Start with a small context such as three to ten results, then evaluate whether larger windows produce enough quality gain to justify their cost. If the feedback model and base retriever agree on nearly every query, the feedback pass may add complexity without value.
A practical pipeline often combines the original results and feedback-search results, then applies a final lightweight reranker or score normalization. This guards against feedback mistakes. If the feedback model misreads the initial context, the original retrieval results can still preserve useful candidates.