Context Precision is a retrieval evaluation metric that measures how well your RAG pipeline’s retriever ranks relevant documents above irrelevant ones. It answers a deceptively simple question: of the chunks you retrieved, are the useful ones at the top of the list, or are they buried beneath noise?
This distinction matters enormously because most LLMs exhibit a strong positional bias — they pay disproportionate attention to the first few chunks injected into the context window. If your retriever places an irrelevant chunk at position 1 and the relevant chunk at position 3, the generator may latch onto the wrong information, producing a less accurate or hallucinated response. Context Precision captures exactly this failure mode.
Within the RAGAS (Retrieval-Augmented Generation Assessment) framework, Context Precision is one of the core retrieval-side metrics, sitting alongside Context Recall. While Context Recall asks “did you find everything you needed?”, Context Precision asks “did you put the good stuff first?” Together they give you a complete picture of retriever quality.
Context Precision is computed as a weighted average of Precision@k across all positions in the retrieved context list. The formula is:
Context Precision@K = (1 / Total Relevant Items in Top K) × Σk=1..K (Precision@k × vk)
Where Precision@k = (number of relevant items in positions 1..k) / k, and vk ∈ {0, 1} is the relevance indicator at rank k (1 if the chunk at position k is relevant, 0 otherwise).
The key insight is the vk term: Precision@k only contributes to the sum when a relevant chunk actually appears at position k. This means that the metric rewards systems that cluster relevant results at the top of the ranking and penalizes those that scatter them among irrelevant filler.
Consider a query “Where is the Eiffel Tower?” with 5 retrieved chunks where the relevance judgments are [1, 0, 1, 0, 0] (relevant, irrelevant, relevant, irrelevant, irrelevant):
Position k: 1 2 3 4 5
Relevant?: 1 0 1 0 0
Precision@k: 1/1 1/2 2/3 2/4 2/5
=1.0 =0.5 =0.67 =0.5 =0.4
Only positions where v_k = 1 contribute:
Position 1: Precision@1 = 1.0
Position 3: Precision@3 = 0.67
Context Precision = (1.0 + 0.67) / 2 = 0.833Now consider the same chunks reordered as [0, 1, 0, 1, 0]:
Position k: 1 2 3 4 5
Relevant?: 0 1 0 1 0
Precision@k: 0/1 1/2 1/3 2/4 2/5
Contributing positions:
Position 2: Precision@2 = 0.5
Position 4: Precision@4 = 0.5
Context Precision = (0.5 + 0.5) / 2 = 0.5Same chunks, same number of relevant results — but Context Precision drops from 0.833 to 0.5 because the relevant items are ranked lower. This is exactly the signal you want: the metric tells you that ranking quality degraded.
RAGAS determines chunk relevance using an LLM-as-a-judge approach. For each retrieved chunk, the evaluator LLM is asked: “Given the question and the ground-truth answer, is this chunk relevant?” The LLM returns a binary yes/no verdict for each chunk, producing the relevance vector used in the formula above.
This approach has important implications:
RAGAS also offers a related metric called context_utilization (sometimes called “Context Precision without reference”) that uses only the question and retrieved contexts — no ground-truth answer required. This variant asks the LLM judge: “Is this chunk useful for answering the question?” It’s less precise but useful when you don’t have labeled data.
from ragas import evaluate
from ragas.metrics import context_precision
from datasets import Dataset
# Prepare evaluation dataset
eval_data = Dataset.from_dict({
"question": [
"What is the capital of France?"
],
"contexts": [
[
"Paris is the capital and largest city of France.",
"France is a country in Western Europe.",
"The Eiffel Tower is a landmark in Paris.",
"Berlin is the capital of Germany.",
"French cuisine is renowned worldwide."
]
],
"ground_truth": [
"The capital of France is Paris."
]
})
# Run evaluation
result = evaluate(eval_data, metrics=[context_precision])
print(result)
# {'context_precision': 0.9} — relevant chunk ranked firstContext Precision is your primary diagnostic tool when users report that your RAG system “knows the answer is in there somewhere” but generates incorrect or incomplete responses. A low Context Precision score (below 0.7) with a high Context Recall score is a classic symptom: the retriever is finding the right documents but ranking them poorly, causing the generator to attend to irrelevant chunks first.
When evaluating hybrid search configurations (e.g., BM25 + dense embeddings with RRF fusion vs. RSF fusion), Context Precision provides a direct comparison of ranking quality. In practice, teams often find that:
Context Precision naturally degrades as you retrieve more chunks (K increases), because additional chunks are more likely to be irrelevant. By plotting Context Precision against K, you can find the sweet spot where you’re retrieving enough relevant material without diluting the context with noise. Most production systems settle on K=3 to K=5 for this reason.
When evaluating whether to switch embedding models (e.g., from text-embedding-ada-002 to text-embedding-3-large), Context Precision on a held-out evaluation set provides a rigorous comparison. Unlike end-to-end answer quality (which is confounded by the generator), Context Precision isolates the retriever’s contribution.
Running Context Precision on a sample of production queries (with human-labeled ground truths) on a weekly cadence helps detect retriever degradation early. Common causes include: