Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents

What it is

ProvenanceGuard is a post-generation verification layer for MCP-based LLM agents, built with MiniLM (source routing) and a DeBERTa NLI model (support scoring), described in Multiverse Computing's paper of the same name. It runs as a black-box add-on over an agent's captured MCP trace (tool outputs + source IDs) — no retraining required.

How it works

Five sequential steps: decompose the answer into individual claims, route each claim to its most relevant MCP source (MiniLM), score whether that source actually supports the claim (DeBERTa NLI), check whether the source matches what the answer states or implies, then emit a per-claim verdict plus a global allow/block decision. Literal values (numbers, dates, identifiers) are checked explicitly rather than passed on plausibility alone. Blocked answers can route through a RARR-style repair step (source-grounded rewrite or fallback), then get re-verified.

flowchart TD
    A[Answer] --> B[Decompose into claims]
    B --> C[Route claim to source - MiniLM]
    C --> D[Score support - DeBERTa NLI]
    D --> E[Check attributed source matches]
    E -->|Pass| F[Allow]
    E -->|Fail| G[Block]
    G --> H[RARR repair attempt]
    H -->|Succeeds| I[Re-verify]
    H -->|Fails| J[Fallback text]

Preconditions / Affected

Requires the agent to preserve a trace that keeps tool outputs tied to distinct source IDs — pooling evidence into one anonymous context defeats the method. Reported results come from a local-model configuration (MiniLM, DeBERTa, local LLM for decomposition); the authors note hosted/cloud models could be substituted but would need separate testing and calibration.

What to do

No install commands are given; the actionable detail is methodological. Teams building MCP agents should keep per-source trace identity intact (not pooled context) so a verifier like this can be applied. Note the performance profile for capacity planning: roughly half a second per answer end-to-end, with NLI and routing calls themselves in the tens of milliseconds — described as modest offline-gate overhead.

Caveats

On a harder multi-source test with several similar sources, block/allow F1 stayed strong (0.846) but exact source identification dropped to 50.3% — the paper flags disambiguating similar sources as unresolved. In the main 361-claim test, 67 claims experts considered supported were nonetheless sent to review/repair (a stated false-block tradeoff from the conservative policy), and one of 139 claims that should have been blocked was missed. In the full-trace repair run, 144 of 173 blocked answers ended in fallback text rather than a substantive rewrite. The NVIDIA NVFlow adoption uses only the source-aware verification stage, not the repair loop.