Closed-World Resolution Against Tool Hallucination in LLM Agents

What it claims

The defense itself is ordinary input validation: check that the tool name is in the registry, then check the arguments against its signature. The author says plainly that the check is not novel. The paper's contribution is measurement, plus the argument that this check must run before any gating layer.

• Invocation surface is the main factor. Fabricated-tool calls cluster on raw-JSON surfaces, where the tool catalog sits in the prompt and a bridge parses the model's JSON. That surface produced 34 fabricated-tool calls. Schema-enforced APIs such as OpenAI function calling and Bedrock Converse produced 3, and only from the weakest open-weight models.
• Model scale does not help. A 675B model hallucinated at the same rate as a 7–8B one.
• The resolver catches every measured case. A gating-only stack executes all 322 hallucinations, and the resolver rejects all of them. On the live MCP surface the resolver rejects 154 of 154. Those 154 fall in classes M1–M3 and include frontier models that were clean on the single-registry surface.

How it works

The resolver (Algorithm 1) takes a call (n, α) and the registry R. It rejects the call at the first failing check:

1. n ∉ R → H1, unresolved tool. This covers fabricated names like wipe_disk and near-misses like read_files.
2. Any key in α that the signature σ(n) does not declare → H2.
3. Any required key in σ(n) that α omits → H2.
4. Any value that fails its type, enum or range → H3, or the detectable part of H5.
5. Otherwise the call is allowed.

The signature types are string, int-with-range, bool, enum and opaque id, each with a required/optional flag. The resolver needs no model access, no fine-tuning and no decoder change.

flowchart LR
  C[Emitted call] --> R0{Rung 0: resolver}
  R0 -->|reject| X1[H1, H2, H3, detectable H5]
  R0 -->|pass| G{Causal gate}
  G -->|reject| X2[H4 off-frontier real tool]
  G -->|pass| V{Contract verifier}
  V -->|reject| X3[Tampered contract]
  V -->|pass| E[Execute]
  E -.-> H5r[Schema-valid H5 residue gets through]

The ordering argument (Prop. 2) runs as follows:

• A gate reads risk and authorization from the tool's contract, and a fabricated tool has no contract.
• A post-gate verifier has no digest to compare against.
• So under a fail-open executor, no check placed at or after the gate can stop H1.
• Adding an existence check to the verifier does not escape this. That check is the resolver, and placing it after the gate brings back the same unsoundness.

For multiple MCP servers, the MCP variant resolves each call to a (server, tool) pair. Its taxonomy has five classes:

• M1: cross-server fabrication
• M2: namespace collision
• M3: shadowing or rug pull
• M4: stale definitions
• M5: cross-server signature borrowing

Evidence

• Live study, single registry: 10 hosted models on two surfaces produced 322 genuine hallucinations. The resolver rejected all 322, and a gating-only stack executed all 322.
• Controlled ablation on scripted calls: the resolver drives each schema-detectable class to 0.00 attack success rate (the fraction of hallucinated calls executed). Honest over-rejection is 0.00. Only the H5 residue survives.
• Synthetic MCP benchmark: a naive host scores 1.00 attack success on all five M-classes, and the resolver scores 0.00.
• HTB leaderboard baselines:
• an allowlist
• a fuzzy-name router
• a JSON-schema validator
• first-provider and highest-trust MCP hosts

The fetched text names these baselines but cuts off before their scores.

How to use it

• HTB is described as installable and deterministic. It produces a single score across H1–H5 and M1–M5.
• It ships adapters for an API-Bank-style registry and for real MCP server manifests that contain naturally occurring collisions and shadows.
• No install command, repository URL or licence appears in the fetched text.

The practical takeaways:

• If your agent parses raw JSON or merges several MCP servers, add a registry-and-signature check before any policy or gating layer.
• If you use a schema-enforced API with a single registry, H1 is largely suppressed at decode time.

Limits

• H5 residue is irreducible. If borrowed arguments happen to satisfy the named tool's schema, no schema check can catch them. That becomes a tool-selection problem.
• Trusted-registry assumption. A wrong signature or missing tool in R is invisible both to the resolver and to the paper's hallucination classifier, because they read the same registry (a "shared oracle"). The 0.00 over-rejection result is therefore true by construction, not independent confirmation. The author says so.
• The propositions are definitional. The author flags that only the ordering result (Prop. 2) is non-trivial.
• Fail-open executor assumed. A strict executor that rejects unknown names already behaves as a degenerate resolver.
• The resolver does not address a compromised registry. That is the contract verifier's job.
• Truncated source. The fetched text stops partway into the evaluation. Per-model results, the ten model names, HTB scores and the MCP live-study breakdown were not available. Check the full paper before relying on per-model figures.