Tool-Calling LLM Agents Under Realistic MCP Faults: Scale Does Not Purchase Fault Tolerance

What it claims

MCP-Injector is a proxy that sits between an MCP client and real MCP servers and injects six kinds of fault into tool calls without modifying the agent or the tools. Its fault mix is weighted by how often each (failure mode, persistence) pair appears in GitHub issues and PRs from public MCP servers. Under that profile, MCP-Bench task-completion score fell from 8.08 to 3.23 out of 10. gpt-oss-120B kept no larger share of its fault-free score than gpt-oss-20B (resilience-ratio model, p = 0.96). Its completion rate under faults was 91.1% against 95.6% for the 20B (difference 4.4 points, 95% CI [−0.2, 9.3]). The authors say agent stacks differ in fault absorption and present this as suggestive evidence that framework error handling matters more than model scale. The paper is accepted at NeurIPS 2026.

How it works

The proxy acts as an MCP server toward the agent and as an MCP client toward the real servers. It hooks call_tool and is switched on or off with one configuration flag. Each call passes through four stages: Observe, Decide, Act, Record. Timing faults and explicit errors are produced on the request side without calling the server. Content faults mutate the real server's response.

flowchart LR
  A[Agent call_tool] --> D{Decide: profile, persistence state, seed}
  D -->|OP0| F[Forward unchanged]
  D -->|OP4 error / OP6 drop| S[Short-circuit, server not called]
  D -->|OP5| W[Sleep, then forward]
  D -->|OP1-OP3| M[Forward, mutate response]
  F --> R[Record log]
  S --> R
  W --> R
  M --> R

The operators are: OP1 random character replacement, OP2 word-level substitution, OP3 type coercion (for example str to int), OP4 MCP error message, OP5 artificial delay and OP6 indefinite hang. OP0 is a passthrough. Persistence is set separately. Permanent fires on every matching call once activated. Transient fires once with an activation probability and then resets. Intermittent holds a fault for a random number of calls. Injection decisions come from a seeded stream, so they are deterministic given the seed and the tool-call history, and every fired fault is logged.

The profile was built in six stages:

1. Collect 1,774 repositories from MCP Market and Awesome MCP Servers in September 2025.
2. Mine up to 200 issues and PRs per repository, which gives 103,166 threads.
3. Filter out duplicates, build-related threads and non-English threads, leaving 57,240.
4. Classify each thread with gpt-4o-2024-08-06 at temperature 0, at a cost of about $800.
5. Check 117 threads by hand: Cohen's κ was 0.93 for failure mode and 0.80 for persistence.
6. Synthesise the profile.

Of 14,829 classified failures, 10,393 were explicit errors.

Evidence

The study ran 1,350 cells: 3 stacks × 2 backbones × 3 conditions × 25 tasks × 3 seeds. The stacks were MCP-Bench's multi-round plan-and-execute loop, the OpenAI Agents SDK and a LangChain/LangGraph pre-built agent. The three conditions were proxy off, OP0 passthrough, and OPP, the 18-rule fault profile. Degradation was measured within each (task, seed) pair as OPP minus OP0.

The 25 tasks are single-server MCP-Bench tasks, given to agents as fuzzy prompts, with distraction servers disabled. Each run had a 3,000 s timeout and a loop bound of 40 rounds or turns (25 supersteps for LangGraph). Runs that hit the timeout, hit the loop bound or raised an unhandled exception count as system-level failures. gpt-4o-mini-2024-07-18 judged each completed run five times with randomized rubric order, and the median is reported. The 1,350 cells took about 58 hours of agent execution plus about 18 hours of judge API time.

How to use it

Code, data and the analysis pipeline are at https://gitlab.com/uniluxembourg/snt/sedan/papers/mcp-injector, archived at https://doi.org/10.5281/zenodo.23573765. A fault profile is a configuration of (operator, persistence, scope) entries with selection weights. The backbones ran at MXFP4 precision with a 16,000-token context window under LM Studio on consumer Apple Silicon. The page gives no install commands and no licence.

Limits

• The profile matches the relative frequency of failure reports, not real operational failure rates.
• GitHub issues over-represent persistent, reproducible bugs, so the authors treat the profile as a stress profile. It is 91.3% permanent, and EXPLICIT_ERROR/PERMANENT alone carries 65.6% of the weight.
• Several parameters are hand-set rather than taken from the data: the 30 s delay, the OP1/OP2 perturbation intensity, d = 0.3, and transient activation of 1.
• End-to-end runs are not bit-reproducible, because local serving and the judge are stochastic.
• Only tool calls are in scope. Prompts, resources, streaming and adversarial faults are excluded.
• The scale finding covers two models from one family, gpt-oss, with no per-model prompt tuning.
• Fault-free cells from earlier campaign rounds ran under tighter limits than the fault-injected cells.
• The fetched text stops at the start of Results, so the per-stack numbers behind the framework claim do not appear on the page.