Many evaluation suites treat all wrong answers as one bucket. That is insufficient for frontier models and long-running agents. A model that reliably chooses the same wrong action has a different risk profile from a model that fails unpredictably across repeated attempts. The first pattern suggests systematic bias toward a wrong objective or policy. The second suggests high variance, brittle reasoning, or unstable trajectories through a long action sequence.
Error incoherence evaluation adapts the bias-variance decomposition to model and agent failures. Instead of asking only “how often is the model wrong?”, it asks “what fraction of the remaining error comes from inconsistent outcomes across repeated samples?” This separates error rate from error shape. A stronger model can have lower total error while its residual failures become more variance-dominated on harder or longer tasks.
The source paper applies this idea to reasoning benchmarks, agentic coding, safety evaluations, and synthetic optimization tasks. Across those settings, longer reasoning or action sequences tended to produce more incoherent failures. For builders, the practical lesson is clear: evaluate repeated runs, report dispersion, and distinguish systematic misalignment from stochastic breakdown before choosing mitigations.
Bias captures failures that point in the same direction across repeated samples. In a multiple-choice benchmark, this might mean the model consistently selects the same wrong option. In an agent task, it might mean repeated runs choose the same bad strategy. Bias-dominated failures are useful to study because they reveal stable prompt, training, reward, tool, or policy problems.
Variance captures failures that differ across samples. A model may know enough to sometimes solve the task, but its trajectory depends heavily on sampling noise, reasoning length, tool order, or early mistakes. Variance-dominated failures are harder to debug with one transcript because the next run may fail differently.
Error incoherence is the share of total error attributable to variance. Values near zero mean errors are mostly systematic. Values near one mean errors are mostly inconsistent. The metric is most meaningful when the target outcome is well-defined, repeated samples are comparable, and the scoring function can represent outcome distance rather than only pass/fail labels.
Accuracy can improve while the remaining failures become harder to anticipate. Error incoherence makes that visible by asking whether failures converge on the same wrong answer or scatter across many wrong outcomes.
Start with tasks that have a clear target: multiple-choice answers, unit-test outcomes, benchmark patches, numeric objective values, or model-written evaluation labels. Run each item multiple times under controlled settings. Keep prompt, tools, scaffold, and environment fixed unless the experiment explicitly compares them.
from collections import defaultdict
import math
def score_answer(answer, target):
return 0.0 if answer == target else 1.0
def decompose_item(samples, target):
errors = [score_answer(s, target) for s in samples]
total_error = sum(errors) / len(errors)
counts = defaultdict(int)
for s in samples:
counts[s] += 1
modal_answer = max(counts, key=counts.get)
bias_squared = score_answer(modal_answer, target)
variance = max(0.0, total_error - bias_squared)
incoherence = 0.0 if total_error == 0 else variance / total_error
return {
"total_error": total_error,
"bias_squared": bias_squared,
"variance": variance,
"incoherence": incoherence,
}The simple code above works for discrete answers, but real evaluations often need richer distance functions. For patches, distance may combine test failures, lint failures, and behavioral assertions. For optimization tasks, distance can be objective loss. For safety evaluations, distance can be a calibrated rubric score. The key is to compute bias from the expected or modal outcome and variance from dispersion around that outcome.
Report incoherence by task hardness, reasoning tokens, action count, tool-call count, and scaffold. The important claim to test is not merely that a model sometimes fails. It is whether longer reasoning and longer action traces shift errors from systematic to inconsistent.
Agent teams often evaluate one impressive run or one failure transcript. Error incoherence pushes them to run repeated trials under the same seedable environment. If failures scatter, invest in checkpointing, verifiers, constrained actions, and recovery policies. If failures converge, inspect prompts, tools, reward signals, and task specification.
Safety teams need to know whether a behavior reflects a stable harmful policy or an unstable breakdown. A systematic refusal failure, reward hack, or harmful preference requires different mitigation than a noisy, nonsensical response that appears only under long reasoning. Both matter, but they should not be counted as the same failure mode.
When comparing two models, include both accuracy and error composition. A larger model may solve more tasks overall while leaving a residual set of failures that is more variable on the hardest items. Similarly, a scaffold may reduce variance by adding verifiers or increase variance by adding too many opportunities for tool-order drift.
Use paired task sets and identical environment seeds when comparing models or scaffolds. Run enough samples per item to estimate dispersion. Store raw transcripts because aggregate incoherence is only a triage signal; debugging still requires inspecting concrete trajectories.
def run_repeated(eval_items, model, scaffold, n=20):
rows = []
for item in eval_items:
samples = []
traces = []
for k in range(n):
result = scaffold.run(model=model, item=item, sample_id=k)
samples.append(result.outcome)
traces.append(result.trace_id)
metrics = decompose_item(samples, item.target)
rows.append({
"item_id": item.id,
"difficulty": item.difficulty,
"action_count_p50": median_action_count(traces),
"trace_ids": traces,
**metrics,
})
return rowsTrack total error rate, severity of wrong outcomes, and error incoherence independently. A low-rate but high-severity systematic failure can be more urgent than frequent low-severity variance. Conversely, high incoherence on long tasks may indicate that single-run demos are unreliable even if average scores look acceptable.
Dashboards should make the mitigation path obvious. Bias-heavy failures should link to representative modal wrong traces. Variance-heavy failures should link to divergent traces for the same item, along with reasoning length, action count, verifier results, and first divergence points.