Rehearse Everything, Remember Nothing: Attic-KV Rehearses What Will Be Read

What it claims

Attic-KV argues that in query-agnostic, compress-once KV caching, what you rehearse matters more than how you score it. The authors call this "a cache keeps what it rehearses." On Qwen3-8B RULER-4K, swapping KVzip+'s attention scorer for KVgrad's gradient attribution adds 13.4 points at 5% keep and 22.0 at 3%. Swapping only the rehearsal for Attic's adds 23.5 and 41.9. The target regime is 3–5% keep, where a fixed memory holds 20× as many caches. At 30% keep it holds only about 3×.

How it works

Documents are processed in 4K-token chunks. For each chunk Cⱼ, Attic builds two rehearsals and scores every KV entry by the max over both. The host's scorer φ is left unchanged.

flowchart LR
  C[4K chunk Cj] --> A[Anchors: keys far from avg key]
  A --> S[Expand to sentences holding anchors]
  S --> R[Rj: content-adaptive length]
  C --> L[Model writes 6 Q&A, answers quote the page]
  L --> Q[H⊕Q]
  R --> M[max of host scores φ]
  Q --> M
  M --> K[Keep top r entries]

1. Anchors. These are the top ⌊r|Cⱼ|⌋ positions by KeyDiff score, which is the negative average cosine to the mean key.
2. Content-adaptive amount. Each anchor is expanded to the full sentence containing it, so mⱼ equals the token count of those sentences. A prose chunk gets a short rehearsal; a fact-dense table gets a long one.
3. Self-generated QA. The model writes 6 questions a reader would likely ask and answers each by quoting the context.

Attic is training-free and plugs into KV², KVgrad, and RestoreKV+. With RestoreKV+ it reuses the released restore tokens without retraining.

Evidence

All results below are on Qwen3-8B, RULER-4K, at 5% keep unless noted.

• Diagnosis. Full-context rehearsal keeps only 5.7–7.1% of each answer's KV entries on 7 of 8 needle tasks, which is roughly chance. Rehearsed positions cover about 5% of the document yet take one fifth to one third of the kept budget.
• Capacity is not the bottleneck. An oracle that rehearses the gold Q+A scores 93.5, against 94.7 uncompressed. Rehearsing the question without its answer scores only 47.0, so the answer must be produced during rehearsal.
• Ablation for the second principle. At an equal average amount, a content-adaptive amount beats a fixed fraction, 78.0 vs 74.6.
• Per task, KVzip+ → Attic:

• At 3% keep: Attic scores 73.4, which is 6.4 above KV² and 41.9 above KVzip+. Uncompressed is 96.5; KVzip+ is 31.5.
• LongBench natural-text tasks: KVzip+ scores 28.6, below methods that rehearse nothing (up to 36.5).
• Hosts: Attic adds up to 17.1 points to KVgrad and up to 28.1 to RestoreKV+. RestoreKV+ with Attic beats every existing method in all 6 settings where restore tokens exist.
• Cost: Compressing a LooGLE document and answering all of its questions takes 126.0 s, vs 154.8 s for KVzip+.

How to use it

The fetched text gives no code, repository, or licence. Adopting Attic amounts to replacing the rehearsal text in an existing rehearsal-scoring pipeline: KVzip+-style normalized attention, KVgrad, or RestoreKV+. Generating the QA pairs requires an LLM generation pass per chunk at compression time.

Limits

• Truncated text. The fetched page cuts off at the start of Section 3. The full results tables, the identities of the "four LLMs," and the LooGLE numbers are not visible here.
• Shown regression. MK2 falls from 73 to 57 versus KVzip+, and MK3 stays low at 30.
• Development-split diagnostics. The diagnostics use the RULER-4K development split, and most per-task figures are for Qwen3-8B only.
• Narrower test coverage. LongBench is tested with two models, against four for RULER.
• Dependence on generated questions. The QA rehearsal depends on the model anticipating the right questions. The page does not separate the QA generation cost from the 126 s figure.