Extractive question answering is the reader stage of a retrieval system: given a user question and one or more evidence passages, the model returns a span that already appears in the source text. Unlike a generative answerer, it does not synthesize new wording by default. It scores possible start and end positions in the passage, chooses the most plausible span, and returns character offsets that can be highlighted or cited directly.
The deepset/roberta-base-squad2 model is a practical example of this pattern. It fine-tunes RoBERTa on SQuAD 2.0, a reading-comprehension dataset that includes both answerable and intentionally unanswerable questions. The unanswerable cases matter in production because a reader should sometimes say “the evidence does not contain the answer” instead of selecting a fluent but unsupported phrase.
In modern RAG stacks, extractive readers are useful when traceability matters more than conversational polish: legal passages, support articles, policy documents, incident runbooks, technical manuals, and search-result snippets. A retriever narrows the corpus to candidate passages; the reader finds the exact answer span inside those passages; the application can show both the answer and the surrounding context.
A reader model consumes a question and context together, usually as a paired sequence: [question] [separator] [context]. Its output is not a free-form response. It produces logits over token positions, one distribution for answer starts and another for answer ends. The best answer is the legal span whose combined start/end score wins after filtering for maximum length and context boundaries.
SQuAD 2.0 extends ordinary span selection with examples where the passage does not answer the question. Training on those examples teaches the model to compare the best candidate span against a null answer score. At inference time, the pipeline needs an explicit threshold or handle_impossible_answer behavior; otherwise it may still return the least-bad span even when no span is actually supported.
Transformer readers have a maximum sequence length. Long documents must be split into overlapping windows so that answers near chunk boundaries are not lost. Parameters such as max_seq_len, max_query_length, and doc_stride determine how much context each model call sees and how much overlap exists between adjacent windows.
Extractive QA turns retrieval evidence into a precise answer span. Because the answer is tied to document offsets, the application can highlight the source text, show nearby context, and reject answers that are not present in the retrieved passage.
The simplest pipeline passes a question and a short context to a model fine-tuned for question-answering. The result includes an answer string, a confidence score, and offsets. In Haystack, the reader component accepts a list of documents and returns extracted answers tied back to document metadata.
from haystack import Document
from haystack.components.readers import ExtractiveReader
docs = [
Document(content="Python is a popular programming language."),
Document(content="Rust emphasizes memory safety without garbage collection."),
]
reader = ExtractiveReader(model="deepset/roberta-base-squad2")
reader.warm_up()
result = reader.run(
query="What language is popular?",
documents=docs,
)
for answer in result["answers"]:
print(answer.data, answer.score, answer.document.id)In a larger RAG system, the reader usually sits after lexical, dense, or hybrid retrieval. The retriever returns top-k passages; the reader evaluates each passage; the application sorts answer candidates by reader confidence, retrieval score, or a combined score. This two-stage design keeps reader inference affordable because the span model only sees a small candidate set.
from transformers import pipeline
qa = pipeline(
"question-answering",
model="deepset/roberta-base-squad2",
tokenizer="deepset/roberta-base-squad2",
)
def answer_from_hits(question, hits):
candidates = []
for hit in hits:
result = qa(
question=question,
context=hit["text"],
max_seq_len=386,
doc_stride=128,
max_answer_len=40,
handle_impossible_answer=True,
)
if result["answer"]:
candidates.append({
"answer": result["answer"],
"reader_score": result["score"],
"retrieval_score": hit["score"],
"source_id": hit["id"],
"start": result["start"],
"end": result["end"],
})
return sorted(
candidates,
key=lambda x: (x["reader_score"], x["retrieval_score"]),
reverse=True,
)Fine-tuning examples provide the question, context, and gold answer offsets. The model learns start and end classifiers over contextual token embeddings. For unanswerable examples, the training target points to the null position or an equivalent no-answer class. The loss combines start-position and end-position errors, so the model learns both where an answer begins and where it should stop.
Support systems often need short answers with citations rather than creative prose. A reader can extract the exact sentence fragment that answers a billing, configuration, or troubleshooting question, then display the source article and highlight the answer span. This reduces the risk of unsupported generated answers.
Policy-heavy teams can use extractive QA to answer questions from handbooks, control catalogs, standard operating procedures, or audit evidence. The model is especially useful when the answer must be traceable to a fixed text and reviewers need to inspect the surrounding paragraph before accepting the result.
A reader can act as a final precision layer for search. Instead of returning whole passages, the system returns the most answer-bearing span from each passage. This is different from a cross- encoder reranker: the reader is optimized to locate an answer, not merely to rank relevance.
Keep the retriever, reader, and final answer policy as separate components. The retriever owns recall, the reader owns span extraction, and the policy layer decides whether to show an answer, ask a clarifying question, or fall back to broader search. This separation makes it easier to tune each stage without hiding threshold decisions inside model calls.
def should_return_answer(candidate, min_reader=0.35, min_margin=0.08):
if candidate is None:
return False
if candidate["reader_score"] < min_reader:
return False
if candidate.get("null_margin", 1.0) < min_margin:
return False
return True
def format_grounded_answer(candidate, document):
start = candidate["start"]
end = candidate["end"]
return {
"answer": document["text"][start:end],
"source": document["url"],
"context": document["text"][max(0, start - 180):end + 180],
}For long contexts, run the reader over overlapping windows and map token offsets back to document offsets. Duplicate spans can be merged by normalized text and source location. If two windows produce different answers, prefer the answer with higher reader score only after checking that both windows came from similarly relevant retrieved passages.
Do not throw away offsets after extracting the answer string. Offsets power highlighting, audit logs, copyable citations, and reviewer workflows. They also help detect failures: if the answer string appears many times in the same document, the offset tells reviewers which occurrence the model actually selected.
doc_stride reduces boundary misses but increases duplicate inference. Tune it against real document lengths and answer-position distributions.