Defense-in-Depth for LLMs: Evaluating Memory Gates Against Activation-Induced and Memory-Induced Sycophancy

What it claims

When an LLM retrieves stored user memories, it can side with the user's stored beliefs over evidence. This is "memory-induced sycophancy." The paper tests two defenses, alone and together:

• Router Gate: a per-memory filter that keeps, rewrites, or drops each retrieved memory.
• Inverse activation steering: subtracting a sycophancy direction from the model's activations.

The filtering result holds. On Llama 3.1 8B at no steering (α=0), Router Gate keeps 43.99% judge-averaged accuracy on MemSyco-Bench. Dropping all memory falls to 13.51%. Router Gate also significantly lowers objective-fact sycophancy against No Defense in 11 of 12 model–judge pairs.

The steering result does not hold. Inverse steering on top of the gate (α=−1.5, Llama) moves sycophancy from 35.80% to 31.32%, but the change is not significant (p=0.08–0.63). The authors conclude that memory filtering is the dependable part and that inverse steering is not shown to add to it.

How it works

Steering vector

• Built from 100 pairs from meg-tong/sycophancy-eval. Each pair is a sycophantic and a truthful completion to a question that states an incorrect belief.
• The vector d is the mean difference of layer-20 residual activations over the answer tokens.
• At inference, α·d/‖d‖ is added at blocks 19, 20 and 21, at the last-token position, via PyTorch forward hooks.
• Decoding is greedy, at most 512 new tokens, in bfloat16.
• The α ladder is {−6, −2.5, −1.5, 0, 1.5, 2.5, 6, 20, 28, 30}. Positive values push toward sycophancy.

Memory defenses. All five share one prompt skeleton and differ only in the memory block:

1. No Defense: memories inserted as plain text (MemSyco baseline).
2. Blanket Rewrite: every memory gets a provenance clause, e.g. "This records a belief the user holds, not an established fact".
3. Router Gate: described below.
4. Blanket Drop: "Retrieved memory: [none]".
5. MemSyco Caution: No Defense plus the benchmark's own caution instruction (MemSyco baseline).

Router Gate. DeepSeek-V4-Flash runs with a frozen prompt at temperature 0 and assigns two labels:

• Each memory: belief / preference / user fact / unclear.
• The question's need: objective / personal-in-scope / personal-out-of-scope / unclear.

A fixed action table maps the pair to Keep, Rewrite or Drop. Rewrite uses the Blanket Rewrite templates.

flowchart LR
  M[Retrieved memory] --> C[Classifier LLM]
  Q[User question] --> C
  C --> T{Action table}
  T -->|Keep| P[Prompt]
  T -->|Rewrite + provenance| P
  T -->|Drop| X[Excluded]
  T -->|unclear| P

The "unclear" label always falls back to Keep.

Evidence

Setup

• Four models: Qwen 2.5 7B, Mistral 7B, Phi-3.5 Mini, Llama 3.1 8B.
• 200 cells (10 α × 5 defenses × 4 models), each generating all 1,550 MemSyco-Bench items.
• No Defense is judged on all items. Defense conditions are judged on a fixed stratified 250-item subsample (50 per task, seed 1729), so comparisons are paired.
• Judges: DeepSeek-V4-Flash, GPT-4o-mini, Gemini 3.7 Flash.

Router Gate at α=0

• Lowers objective-fact sycophancy versus No Defense on every model (Holm-corrected McNemar).
• p<0.05 in 11 of 12 model–judge pairs, and p<10⁻³ on Llama. The one miss is Qwen under Gemini (p=0.052).

Blanket Drop

• Gives the lowest pooled sycophancy, mostly because tasks that test memory misuse cannot fail when no memory is present.
• Its memory–evidence-conflict score falls to 0–44%.

Inverse steering (Llama + Router Gate, 149 paired items)

• Across all 600 tests comparing α<0 with α=0, none shows a significant reduction after Holm correction.
• The two tests that do survive are increases.
• Without the gate, α=−1.5 raised Llama's objective-fact sycophancy by 8.2 points under Gemini (uncorrected p=0.004).

Positive steering

• Significantly raises objective-fact sycophancy on Phi (α=28) and Mistral (α=2.5) under all three judges.
• Does so on Qwen under two of three judges, and not on Llama.
• Router Gate stays below No Defense in every cell that did not crash.

Router Gate vs Blanket Rewrite. The gate scores lower in 24 of 32 model–α rows, but the judges disagree on the ordering in 15 of those. The authors do not claim the gate beats Blanket Rewrite.

How to use it

The repository is https://github.com/StellarDrift15/AlgoverseRRPR. It contains:

• the steering hook, prompt builder and all five defenses;
• the gate classifier prompt and action table;
• frozen steering vectors;
• all 200 generation reports;
• per-row judge logs.

The per-row layer-sweep logs are not included. The page gives no install commands and no licence. Generation ran on Colab A100-40GB in bfloat16.

Limits

• No steering controls. The authors ran neither a random-direction control nor a true/false agreement control. Negative-α reductions may therefore reflect generic disagreement rather than a sycophancy-specific direction.
• Thin sampling. Each item gets one greedy generation, and each model gets one steering vector. Seed and vector variance are not reported.
• Wide intervals. The 250-item defense subsample gives about ±8 points of uncertainty on pooled rates.
• Judge sensitivity. The choice of judge shifts absolute rates by up to 45 points in low-confidence cells.
• Judge overlap with the gate. DeepSeek-V4-Flash is both the gate classifier and one of the judges.
• Steering instability. Mistral 7B crashes at |α|≥6 and Llama 3.1 8B at α≥20. Phi-3.5 Mini's high-α outputs are mostly truncated at 512 tokens.
• Prompt scope. The MemSyco conversation transcript is omitted from all prompts.
• No comparison with published filters. MemGate was not run. Router Gate is a separate, simpler method.