Context precision and context recall are the two halves of "did the retriever do its job," reframed in the language Ragas uses. They map onto classical IR precision and recall, with two practical wrinkles: the labels can come from a gold answer instead of per-chunk relevance, and the scoring can run without graded human labels. The combination makes the metrics cheap to run at PR time, which is what matters in practice.
What the two metrics measure
Context recall asks: of everything in the gold answer, what fraction is covered by the retrieved context? Decompose the gold into atomic claims, then check whether each claim can be inferred from the retrieved chunks. High recall means the retriever surfaced what the answer needs; low recall means the model would have to invent something [1] [2].
Context precision asks the dual: of the chunks returned, are the relevant ones ranked above the noise? Ragas runs an LLM judge over each chunk, conditioned on the query and an answer: the gold reference for ContextPrecision, the generated response for ContextUtilization. It then averages precision@k over the ranks where the judge said the chunk was useful [3] [4]. That makes the score rank-aware: noise ranked below every relevant chunk leaves it unchanged, while noise ranked above a relevant chunk pulls it down. High precision means the relevant chunks lead the prompt; low precision means the generator wades through unrelated text first.
The pair is the precision-recall tradeoff at the retrieval layer. Higher K raises recall and lowers classical precision. Reranking typically raises both. Better chunking can raise either, depending on whether the misses were "answer not in the index" (recall) or "right chunk near similar but wrong chunks" (precision).
| Metric | Question it answers | Needs a reference answer? | Classical analogue |
|---|
| Context recall | Of the gold answer, how much is covered by the retrieved context? | Yes | Recall@K |
| Context precision | Of the retrieved chunks, are the relevant ones ranked first? | ContextPrecision: yes. ContextUtilization: no, it uses the generated response | Precision@K, rank-weighted |
Figure: Context recall and context precision score the same retrieval from opposite directions: recall decomposes the gold answer into atomic claims and checks coverage by the retrieved context (1/1 = 1.0), while precision judges each retrieved chunk and rewards relevant chunks that rank high. The one relevant chunk sits at rank 3 of 3, so the score is 1/3, about 0.33. Move it to rank 1 and the same three chunks score about 1.0.
Why "context" instead of "chunk"
Classical IR uses graded per-document labels. Ragas substitutes an LLM judge over chunk-claim pairs because graded labels are expensive. The substitution costs accuracy: judge noise is the dominant source of measurement error on these scores [5]. The practical answer is to use the LLM-judge implementation for PR-time signal and calibrate the judge against a hand-labeled set of fifty to a hundred queries before trusting absolute numbers. Calibration question: does the judge agree with the principal expert above 90 percent on a held-out set? If yes, the score is usable.
A runnable example
# context_pr.py
from openai import AsyncOpenAI
from ragas.llms import llm_factory
from ragas.metrics.collections import ContextRecall, ContextPrecision
# judge model: swap for the one you run
llm = llm_factory("gpt-4o-mini", client=AsyncOpenAI())
recall = ContextRecall(llm=llm)
precision = ContextPrecision(llm=llm)
sample = {
"user_input": "When did the company file for IPO?",
"reference": "The company filed for IPO on March 12, 2024.",
"retrieved_contexts": [
"The 2023 annual report covered fiscal year ending December 31, 2023 ...",
"Acme Corp was founded in 2009 by two engineers from Stanford ...",
"Acme Corp filed its S-1 prospectus with the SEC on March 12, 2024 ...",
],
}
print("context_recall", recall.score(**sample).value)
print("context_precision", precision.score(**sample).value)
Ragas 0.4.3, released 2026-01-13 and still the latest release on PyPI as of 2026-10-02, deprecates the older entry points [6]. evaluate() warns and points to the @experiment decorator, and importing metric instances such as context_recall from ragas.metrics warns that they move to ragas.metrics.collections and go away in v1.0 [7]. The snippet above uses the replacement. Each call scores one sample: loop over your rows, call the async ascore, or pass a list of the same dicts to batch_score.
The relevant chunk is last, behind two unrelated ones. Context recall is high (the IPO date is covered). Context precision is one-third if the judge marks only that chunk useful: precision@3 at the one relevant rank is 1/3, divided by one relevant item. Move the chunk to rank 1 and the same call scores about 1.0, noise and all; at rank 2 it scores 0.5. Reading the two numbers together is the diagnostic. Both high: the retriever is dialed in. High recall, low precision: the relevant chunks sit below noise, so add a reranker or fix the scoring. Cutting K saves tokens but leaves this score unchanged when the noise already ranks below the relevant chunks (classical Precision@K would improve; this score will not). Low recall, high precision: raise K, revisit chunking, or check the embedding model. Both low: the retrieval architecture is wrong for the corpus shape; the next move is usually hybrid search or a different chunking strategy.
Which one needs a reference answer?
Context recall requires a reference answer. Context precision comes in two variants: ContextPrecision needs one, and ContextUtilization swaps it for the generated response [4]. The practical consequence: the response-based variant is cheaper to run, because it can score unlabeled production traces while recall needs a labeled set [8]. The right move is to compute it over a sliding window of production traces (live drift signal) and recall on a static labeled set (stable coverage signal).
NOTE
The two precision variants answer different questions. The judge asks whether each chunk was useful in arriving at the given answer. With the gold reference as that answer, the score is a retrieval-only signal. With the generated response, a poor generation can lower the score even when retrieval was fine. If your dataset has gold answers, use ContextPrecision. Treat ContextUtilization as the fallback for traces without labels, and do not compare its numbers with ContextPrecision.
Mapping to classical IR
The rough mapping: context recall is recall at the claim level, with a generative judge replacing graded per-document labels. Context precision is the analogue of mean average precision restricted to the top-K window. Ragas also ships LLM-free ID-based variants, IDBasedContextPrecision and IDBasedContextRecall, which compare retrieved_context_ids against reference_context_ids. The precision one is the unordered classical Precision@K: retrieved IDs found in the reference IDs, over retrieved IDs [4]. As of 0.4.3 they still import from ragas.metrics, which warns, and have no ragas.metrics.collections counterpart yet. For the classical, LLM-free counterparts, see Recall@K and Precision@K; when either the classical or the Ragas score drops, retriever debugging is the elimination path. Neither Ragas metric is a literal substitute for the IR version, so do not compare Ragas numbers across versions or implementations without recalibrating [9]. Treat them as internal control charts, not leaderboard entries.
Thresholds in practice
For a first calibration, aim for context recall above 0.85 and context precision above 0.5. Below those, the generator either makes things up (low recall) or wades through noise (low precision). Above them, further retrieval tuning produces diminishing returns and the next bottleneck is usually the prompt or the model. Tune up for products with unambiguous gold answers (factoid lookup, structured extraction) and down for products where the gold itself is fuzzy (multi-document synthesis, subjective queries).