Does Document Structure Help Dense Retrieval? A Placebo-Controlled Ablation of Four Mechanisms Across Two Corpora

What it claims

Across 11 conditions on two corpora, where you cut chunks matters more than what you prepend to them. Moving from fixed windows to bare structure-aligned chunks (A0→A2) is worth two to four times the gain from then adding the real heading path (A2→A3).

• Anthropic-style contextualization loses. LLM contexts on fixed windows (A1) rank below bare structure-aligned chunks, with no prefix at all, on both corpora. The authors say that under realistic token budgets, where prefix tokens count against the context window, the prepended contexts are strictly harmful.
• Shuffled breadcrumbs don't help. Prepending another document's heading path gains nothing over no prefix, or actively hurts.
• Induced structure is good enough on papers. On QASPER, structure induced by a 26B LLM is statistically as good as the authors' gold headings.

How it works

The prefix format is [Title › Section › ... ] \n\n content. Relevance is scored only on content character offsets, so a prefix can never create spurious relevance.

• Size matching. Each document's fixed-window size is set to the mean length of that document's induced chunks. Realized means differ by ≤4 tokens:
• Wikipedia-FA: 255.4 vs 259.6
• QASPER: 244.9 vs 249.8

Windows snap to sentence boundaries. Structural chunks are split above 350 content tokens and merged below 60.

• Placebo. Real heading paths from other documents, bucketed by depth and permuted toward a cross-document derangement with a fixed seed. Format, depth and length all match a real breadcrumb.
• Hierarchical arm (A4).
• Section centroids are the renormalized mean of the same chunk vectors flat search uses.
• Stage 1 keeps the top 5 sections, where a section is a document plus a top-level heading.
• Stage 2 ranks chunks inside those sections and backfills from the global ranking if fewer than k remain.
• Structure induction. Gemma-4-26B-A4B (MoE) runs locally and sees paragraphs with headings stripped. It returns (title, level, paragraph-range) tuples that must tile the document exactly. Failures get one self-correction round, then a deterministic repair.
• Wikipedia-FA: 198/200 documents usable, 116 needed repair.
• QASPER: 1,579/1,585 usable, 590 needed repair.
• Only 78.9% (Wikipedia) and 75.8% (QASPER) of induced spans are byte-identical to gold spans.
• Shared LLM. The same Gemma model generated the contexts and the synthetic queries.

flowchart LR
  A1[A1: LLM context on fixed windows] -->|C1| A3
  P[P: shuffled heading path] -->|C2| A3
  G3[G3: gold heading tree] -->|C3| A3
  A4[A4: route to top-5 sections] -->|C4| A3
  A3((A3: structure chunks + real path, flat search))

All four pre-registered contrasts are measured against A3.

Evidence

The metric is cov-nDCG@10, an α-nDCG-style score with partial credit for multi-span evidence. The primary cell uses a strong embedder (the page does not name it) and no reranker. Significance comes from a document-clustered bootstrap with Holm correction.

• Effect sizes are small: dz 0.06–0.11.
• On Wikipedia-FA, C4 is the only contrast the conservative secondary model also flags.
• First-stage section recall explains the C4 loss. It is 0.60–0.68 on the 200-document corpus and collapses to 0.17–0.20 on the 1,508-document corpus.
• On QASPER, adding a cross-encoder reranker recovers nearly all of the C4 loss (−0.015 → −0.004).

Limits

• Size of the gains. Gains are a few hundredths of nDCG, so whether they justify re-chunking a pipeline is a cost call.
• Only naive routing was tested. The hierarchical arm routes to top-5 top-level sections. The page cites parent–child reranking as the hierarchy that does help, and that was not tested here.
• Synthetic queries on Wikipedia. The Wikipedia queries were generated by the same Gemma model, though they were leakage-filtered and their gold spans were chosen by the pipeline, not reported by the LLM.
• Scope of the pre-registration. It covers one cell (primary metric, strong embedder, no reranker). Results for other embedders appear in sections not included in the fetched text.
• Truncated text. The fetched page stops partway through corpus construction. The appendix condition table, the full results, and the section-recall analysis were not available.