Protection Is Not One Event: A Survey of RAG Protection Research by Measured Endpoint

What it claims

The survey sorts 118 RAG defense papers by the event their primary metric actually measures, not by the defense mechanism or the goal stated in the abstract. The central finding is that "protection" papers rarely measure the same thing. Only 21 of 118 (17.8%) score whether protected content is emitted or reconstructed. The largest family, integrity and attribution, holds 47 records.

How it works

The taxonomy has seven families and 17 subfamilies. Each paper is assigned one primary endpoint family, while its mechanisms and pipeline stages can receive several labels. Record counts per family:

• A, emitted wording (verbatim or near-verbatim): 8
• A-pii, emitted identifiers (direct or quasi): 6
• B, source recovery outside generation (from embeddings, index, cache or parameters): 7
• C, formal or statistical proxy (DP, RDP, certified bound, perplexity): 6
• D, access, retrieval or answer suppression: 26
• E, corpus-use detection (watermark or canary, membership, ownership): 18
• F, poisoning, attribution, prompt injection, removal: 47

Family G is reserved for papers whose reported results do not establish a specific event. Only A, A-pii and B count as observed disclosure. Family C reports a bound or proxy, not leaked text.

The assignment rule changes where some papers land. An unlearning method evaluated only by refusal rate goes under D, not removal. A test of whether deleted data can be rebuilt from residual index state goes under B.

To be included, a paper had to evaluate a RAG system, intervene on its query, corpus, index, context or output, and report a source-verified protection or attribution metric. Attack-only papers, benchmarks, surveys and untested proposals were tracked separately. Utility-only reporting did not qualify.

For Families A and A-pii, the survey groups defenses into four intervention points:

• retrieval-side distance thresholding
• context rewriting or extractive compression
• constrained decoding: DP noise on next-token probabilities, backtracking on entity matches, and dual-path drafting with a restricted reference channel
• two-stage input and output filters for PII

Evidence

The search covered arXiv, OpenAlex and the ACL Anthology through July 17, 2026. It returned 11,972 hits, which deduplicated to 8,978 records. Screening left 925 candidates, and 824 full texts were recovered and double-coded.

Of the 118 core records, 112 came from the main pool. The other 6 came from a double-screen of 1,200 excluded records. The authors estimate about 27.2 additional qualifying studies exist beyond their pool.

Coders filled 34 fields per record. Agreement was κ=0.867 on the primary family, with a median κ of 0.94 across fields (range 0.56 to 1.00). Every cited operating point was checked against the source PDF.

Retrieval reranking and context rewriting each appear across five or six endpoint families. Among records labelled copyright or ownership, 73.3% use corpus-use detection as their primary endpoint, and none measure emitted expression.

How to use it

The authors release an 8,024-row de-identified label matrix and scripts that recompute agreement for every field. Appendix D.2 lists per-study operating points. An engineer comparing defenses can use the matrix to check which event a cited "protection" result actually measured before pooling or comparing numbers. No repository URL or licence appears in the fetched text.

Limits

Proportions, Wilson intervals and permutation p-values describe the 118-record pool, not the whole field. The fetched text stops partway through Section 4.1. Several items are therefore not visible:

• Table 4, which shows endpoint families moving in opposite directions under the same defense
• the reporting-quality audit
• the testbed setup in Appendix C, beyond the 8-gram guard on Qwen3-8B
• the analysis of datasets and threat models