MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI
MemSentry is a proposed pre-commit evaluation framework for agentic AI systems that intercepts proposed writes to persistent memory and issues Accept/Review/Quarantine decisions by jointly scoring source trust, semantic risk, attack radius over a component-dependency graph, access risk, and a signed security-posture delta. The authors evaluate it against 1,000 GPT-4-generated scenarios using four pluggable semantic classifiers (Regex, TF-IDF+SVM, SBERT+LR, SetFit), with SBERT+LR reaching 91.7% accuracy while all methods catch 100% of external quarantine-class threats; the paper also documents a key limitation — surface-similar but oppositely-intentioned insider operations remain hard to distinguish semantically. It targets the persistent-memory-poisoning attack surface distinct from prompt injection, where a single malicious memory write (e.g., "suppress alerts from this IP") can silently persist and influence future agent actions like a SOC assistant's alerting or access decisions.
What it is
MemSentry is a proposed (not released) pre-commit gate for persistent-memory writes in agentic AI, from an arXiv preprint (2609.08747). The summary names the five scored factors; the paper's actual value is the concrete, deterministic decision logic and thresholds an implementer would need. Note it is a paper with a synthetic evaluation (20-asset random dependency DAG, 10×20 access matrix, 1,000 GPT-4-generated scenarios) — no code, no real deployment.
How it works
Each proposed write runs a 9-step pipeline producing a signed security delta δ(O) and a risk score R, then hits rule-based routing. Two mechanisms distinguish it from the summary's high-level description:
Trust is a fixed 4-value mapping, not learned: external-unverified T=0.0, external-verified T=0.5, internal-unverified T=0.5, internal-verified T=1.0.
Semantic categories carry hardcoded risk scores (the classifier just assigns a category): security_strengthening −0.30, preference 0.05, schedule 0.05, operational_fact 0.10, trust_modification 0.70, alert_suppression 0.80, policy_override 0.80, conditional_trigger 0.85, credential_storage 0.90, authority_claim 0.90. Negative-risk (strengthening) operations do not inflate R.
A persistence factor f_pers ∈ {0.3, 0.6, 1.0} (temporary / bounded / indefinite) scales projected impact — a permanent write is amplified 3.3× over an auto-expiring one.
The explanation record is generated deterministically from templates, not by an LLM — identical inputs always produce identical outputs, for auditability.
Decision rules (the part that matters)
flowchart TD
O[Proposed write] --> E{External unverified?<br/>T=0}
E -->|yes| Q[Quarantine — regardless of content]
E -->|no| D{delta < -tau_delta<br/>or R >= tau_Q?}
D -->|yes| Q
D -->|no| A{Internal T>tau_T AND<br/>authorized AND low r_sem<br/>AND benign category?}
A -->|yes, but crit >= 0.85| R2[Review]
A -->|yes| Acc[Accept]
A -->|no| R2Two rules are worth internalizing: (1) any external-unverified source (T=0) is quarantined regardless of content — this is what yields the summary's "100% external quarantine" number; it is a trust-boundary rule, not semantic detection. (2) A verified insider (T=1) is never auto-quarantined — suspicious writes route to human Review instead. This is by design, and it is why the insider-intent limitation the summary mentions actually bites: the semantic classifier is the only thing standing between a compromised insider and an Accept.
Caveats
Best result is SBERT+LR at 91.7% accuracy / 0.908 macro-F1 (summary omits the F1). Evaluation is entirely on GPT-4-generated scenarios over a random synthetic DAG — no real agent, no real memory store. The authors themselves flag that surface-similar, opposite-intent operations defeat all four classifiers, and propose LLM-based intent reasoning as future work.
Technologies named here
- Agentic AI systems with persistent memory
- MemSentry
- SMSR
- Agentic MailBot
- MemAudit
- GPT-4
- Autonomous SOC agent
- MEMSHIELD
- MIVL
- A-memguard
- Regex
- TF-IDF+SVM
- SBERT+LR
- SetFit
- LangChain
Techniques named here
- Persistent Memory Poisoning
- Alert suppression via memory poisoning
- Privilege escalation via memory poisoning
- Trust relationship / policy override via memory
- Authority claim / conditional trigger memory writes
- Prompt Injection
- Coordinated multi-entry memory poisoning
- Context-dependent memory poisoning
Threat actors named here
- External Adversary
- Compromised Insider
Related in AI Security
