Skip to content

Back to the catalog

arxiv.orgpaper

MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI

MemSentry is a proposed pre-commit evaluation framework for agentic AI systems that intercepts proposed writes to persistent memory and issues Accept/Review/Quarantine decisions by jointly scoring source trust, semantic risk, attack radius over a component-dependency graph, access risk, and a signed security-posture delta. The authors evaluate it against 1,000 GPT-4-generated scenarios using four pluggable semantic classifiers (Regex, TF-IDF+SVM, SBERT+LR, SetFit), with SBERT+LR reaching 91.7% accuracy while all methods catch 100% of external quarantine-class threats; the paper also documents a key limitation — surface-similar but oppositely-intentioned insider operations remain hard to distinguish semantically. It targets the persistent-memory-poisoning attack surface distinct from prompt injection, where a single malicious memory write (e.g., "suppress alerts from this IP") can silently persist and influence future agent actions like a SOC assistant's alerting or access decisions.

TL;DR A pre-commit trust/risk/blast-radius scoring framework (MemSentry) to Accept/Review/Quarantine agentic AI memory writes, aimed at defending against persistent memory poisoning attacks.
Document

What it is

MemSentry is a proposed (not released) pre-commit gate for persistent-memory writes in agentic AI, from an arXiv preprint (2609.08747). The summary names the five scored factors; the paper's actual value is the concrete, deterministic decision logic and thresholds an implementer would need. Note it is a paper with a synthetic evaluation (20-asset random dependency DAG, 10×20 access matrix, 1,000 GPT-4-generated scenarios) — no code, no real deployment.

How it works

Each proposed write runs a 9-step pipeline producing a signed security delta δ(O) and a risk score R, then hits rule-based routing. Two mechanisms distinguish it from the summary's high-level description:

Trust is a fixed 4-value mapping, not learned: external-unverified T=0.0, external-verified T=0.5, internal-unverified T=0.5, internal-verified T=1.0.

Semantic categories carry hardcoded risk scores (the classifier just assigns a category): security_strengthening −0.30, preference 0.05, schedule 0.05, operational_fact 0.10, trust_modification 0.70, alert_suppression 0.80, policy_override 0.80, conditional_trigger 0.85, credential_storage 0.90, authority_claim 0.90. Negative-risk (strengthening) operations do not inflate R.

A persistence factor f_pers ∈ {0.3, 0.6, 1.0} (temporary / bounded / indefinite) scales projected impact — a permanent write is amplified 3.3× over an auto-expiring one.

The explanation record is generated deterministically from templates, not by an LLM — identical inputs always produce identical outputs, for auditability.

Decision rules (the part that matters)

flowchart TD
    O[Proposed write] --> E{External unverified?<br/>T=0}
    E -->|yes| Q[Quarantine — regardless of content]
    E -->|no| D{delta < -tau_delta<br/>or R >= tau_Q?}
    D -->|yes| Q
    D -->|no| A{Internal T>tau_T AND<br/>authorized AND low r_sem<br/>AND benign category?}
    A -->|yes, but crit >= 0.85| R2[Review]
    A -->|yes| Acc[Accept]
    A -->|no| R2

Two rules are worth internalizing: (1) any external-unverified source (T=0) is quarantined regardless of content — this is what yields the summary's "100% external quarantine" number; it is a trust-boundary rule, not semantic detection. (2) A verified insider (T=1) is never auto-quarantined — suspicious writes route to human Review instead. This is by design, and it is why the insider-intent limitation the summary mentions actually bites: the semantic classifier is the only thing standing between a compromised insider and an Accept.

Caveats

Best result is SBERT+LR at 91.7% accuracy / 0.908 macro-F1 (summary omits the F1). Evaluation is entirely on GPT-4-generated scenarios over a random synthetic DAG — no real agent, no real memory store. The authors themselves flag that surface-similar, opposite-intent operations defeat all four classifiers, and propose LLM-based intent reasoning as future work.

Technologies named here

Techniques named here

  • Persistent Memory Poisoning
  • Alert suppression via memory poisoning
  • Privilege escalation via memory poisoning
  • Trust relationship / policy override via memory
  • Authority claim / conditional trigger memory writes
  • Prompt Injection
  • Coordinated multi-entry memory poisoning
  • Context-dependent memory poisoning

Threat actors named here

  • External Adversary
  • Compromised Insider