Skip to content

Back to the catalog

When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime
arxiv.orgpaper

When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime

An eight-week longitudinal case study of a production personal-assistant LLM agent runtime (40 scheduled jobs, 8 LLM providers, a tool-governance proxy, and a memory plane) that documented 22 incidents and derived a five-class taxonomy of silent failures: environment/platform quirks, design-assumption mismatches, error swallowing, chained hallucination, and operational blind spots. It introduces "fail-plausible" failures, where an LLM fabricates a fluent, believable narrative to mask an underlying error rather than surfacing it, and finds that audits mostly catch regressions rather than preventing failures, with incident latency tied to failure mechanism rather than code complexity.

TL;DR A production-agent postmortem study taxonomizes silent LLM agent failures and names the "fail-plausible" failure mode where errors are hallucinated into plausible narratives.
Open original ↗
Document

What it adds beyond the summary

The summary names the five-class taxonomy and the "fail-plausible" idea; the page supplies the concrete system, the numbers, the discovery-channel data, the defense framework, and the methodology — none of which the summary carries.

The system under study. Public repo openclaw-model-bridge: two-layer middleware connecting self-hosted and commercial LLMs to OpenClaw (a personal-agent framework bridging WhatsApp and Discord), in continuous production on a single macOS host since March 2026. Three-plane design — control plane (tool-governance proxy, 90 invariants / 827 checks / 23 meta-rules / 14 mechanized scanners / daily audit cron, convergence engine, circuit breakers), capability plane (8-provider adapter, Qwen3-235B primary + Qwen2.5-VL multimodal, capability-scored fallback chains), memory plane (~1,100+ note KB with local-embedding RAG, daily "dream"/"digest"/"deep dive" synthesis jobs). Defended by 4,286 unit tests in 121 suites. Operated by one human owner plus Claude as an AI engineering collaborator; the runtime itself is Qwen3-class — so the paper is also a data point on AI-assisted operation of an AI system, and it flags that Claude co-wrote the postmortems and drafted the manuscript.

The canonical fail-plausible incident. An HTTP 400 error page was captured into a cache by a logging bug; the downstream LLM, seeing error strings where data should be, fabricated an industry analysis about a "Hugging Face platform crisis" and pushed it to the user as a routine insight digest. No detector fired; every test stayed green. Class D reframes hallucination as a systems property — in all 4 fabrication incidents the model behaved as trained (fluent completion over context); the failure was that the system delivered polluted context. Defense is therefore system-side: stderr discipline, alert stripping before context assembly, provenance/source-credibility labeling, layered literal anti-fabrication guards.

Key quantified findings

  • Discovery channel: ~70% of silent failures were caught by a human reading pushed output, not by tests, health checks, or audits (which stayed green). Unit-test catch rate for this class ≈ 0%.
  • Audit scorecard: across 15 audited incidents, 0% ex-ante prevention, 87% ex-post regression blocking — "audit is a regression engine, not a prediction engine." (The summary states this qualitatively; here are the numbers.)
  • Latency: 13 hours to 60 days of silence, correlating with mechanism, not code complexity; the longest-lived failures lived in seams between components where no test runs.
  • Meta-pattern: MR-4 "silent-failure-is-a-bug" manifested ≥28 times across 22 incidents, at a roughly constant rate even as defenses accumulated — in new forms each time. Basis for treating silent failure as a bug class.
  • One sabotage-validation finding: 67 governance checks had been silently executing empty strings for months (an unvalidated detector is indistinguishable from a vacuous one).

Method worth borrowing

Every postmortem follows a mandatory in-repo protocol requiring, before any fix: a full causal-chain diagram; a three-layer root cause — trigger / amplifier / concealer; per-minute timeline reconstruction; condition-combination ("why now") analysis; and feeding new invariants into the governance ontology. Taxonomy is built by mechanism (how the error evaded observation), not location — a location-oriented draft was discarded because one mechanism-level defense (e.g., a repo-wide scanner for a suppression idiom) immunizes every location at once. The paper also reports a "Sunset Law": retiring complexity outranks adding protection (seam reduction over defense accretion).

Limits

Single-system longitudinal case study, self-selected corpus. Classification done by the two operators (human + AI) with no independent annotators and no inter-annotator agreement reported — contrast MAST (κ = 0.88), which the paper cites as the complementary horizontal study. Selection requires a completed postmortem, so failures still silent today are absent by construction (acknowledged surviving-silence bias). This is a June 2026 Draft v0.3 by an independent researcher who owns the system under study; page text cuts off at Class A, so the per-class detail beyond the fail-plausible example isn't fully shown.

Techniques named here

  • Fail-plausible failure mode
  • Five-class mechanism-oriented silent-failure taxonomy

Related in agents