Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability

What it claims

Intervention-based faithfulness is the score that both drives and judges automated circuit discovery. That score can prefer an equal-size circuit that reproduces the model worse. The paper shows this happens even without any discovery algorithm, using hand-edited reference circuits. The choice of ablation strongly affects how often it happens:

• Zero replacement is by far the worst. Across 14 controlled tasks, KL candidate–candidate misranking is 3.7% under resampling, 9.3% under mean replacement, and 33.3% under zero replacement.
• Changing the metric does not fix it. Swapping KL for logit difference gives 6.2% / 7.7% / 19.0% across the same three ablations. Probability difference gives 5.0% / 8.8% / 25.1%.
• Under resampling, KL is the most reliable of the three. For reference-vs-candidate pairs it misranks 3.0%, against 6.1% for logit difference and 6.5% for probability difference.

How it works

• Behavioural ground truth (Q). Q is agreement with the intact model's answers on held-out prompts, including the model's mistakes. It is always measured under ordinary resampling with fixed donor draws. Greater-Than is the exception and uses semantic accuracy instead.
• Faithfulness (F). The main score is negative KL on validation prompts. Under resampling it is averaged over three shared donor draws.
• Misranking. A pair is misranked when F prefers the circuit with lower Q. Ties are counted as correct rankings.
• Size control. Every pair compared has the same number of components.
• Controlled candidates. Retained components of a reference circuit are swapped for an equal number of excluded ones. Edit sizes are one component, then 5%, 10%, 20%, 50% and 75% of the circuit, with ten seeds per size.
• Task suites. The human-studied suite is IOI, Greater-Than, Docstring and Acronym. The semi-synthetic suite is ten InterpBench tasks.
• Repair test. For 100 persistent misrankings, the authors restore selected signals from the intact model's execution. Circuits, Q scores and donor assignments stay fixed.

Evidence

Results depend heavily on task and ablation (Table 2, KL):

Edit size does not predict failure.

• On Greater-Than with zero replacement, every reference-vs-candidate pair in the 50% band is misranked.
• On Docstring with zero replacement, the reference-vs-candidate rate falls from 100% at the smallest edits to 0% in the two largest bands.

Discovery outputs fail more often. For EAP, EAP-IG, ACDC and Edge-SP under resampling, candidate–candidate misranking is 9.4%–41.2%. That is well above the 3.7% seen for controlled edits. Misranking is also higher on the human-studied tasks than on InterpBench, so passing InterpBench is not reassurance.

Repair works in most cases. At least one partial restoration fixed both the validation and test ranking in 96 of 100 cases.

Limits

• "Better" is defined by resampling. Q is fixed to ordinary resampling, so the comparison measures agreement with the intact model under that one ablation.
• No global optima. The paper tests pairwise rankings only; it does not compute the best circuit under either F or Q.
• Restoration is a diagnostic, not a selection method. It supports context distortion as one contributor to misranking.
• Truncated source. The fetched text cuts off in Section 3. The per-method discovery breakdown and the restoration procedure are known only from the abstract and introduction.