Decompose, Estimate, Select: Coverage-Aware Context Construction for Long-Form RAG

What it claims

The pipeline selects RAG context by predicted subquestion coverage rather than independent relevance. On TREC RAG 2024 (56 queries, BM25 top-100 pool, k=10), it lifts α-nDCG@10 from 0.625 to 0.702 with no loss in P@10 or nDCG@10. The gold-nugget bound of 0.718 only measures decomposition quality. Selecting with the true document-nugget matrix (oracle greedy) reaches 0.887, so most of the remaining gap sits in the coverage estimator, not in the subquestions. The paper's second finding matters for anyone tuning retrieval alone. Oracle selection covers far more nuggets in the context, yet the answer conveys no more of them than with the estimated selection.

How it works

flowchart LR
  Q[Query] --> D[LLM: 10 subquestions]
  Q --> B[BM25 top-100]
  B --> E[Coverage bi-encoder rerank]
  D --> C[Cross-encoder coverage matrix]
  E --> C
  C --> G[Greedy max-marginal coverage, k=10]
  G --> A[Generator answer]
  A --> S[Self-critique: underused docs]
  S -->|same documents| A

• Bi-encoder. The bi-encoder is a relevance-trained ModernBERT, further trained with InfoNCE (temperature 0.02). The anchor is the query plus a simulated covered state. Positives add uncovered subquestions. Negatives are redundant or irrelevant documents, with redundant ones always included. The authors say this reproduces CoveR's objective on public data and is not their contribution.
• Coverage estimator. A ModernBERT cross-encoder with 1024-token input scores each (document, subquestion) pair. Scores are binarized at τ=0.3.
• Greedy selection. Each step adds the document covering the most uncovered subquestions. Ties break by bi-encoder score. Once no candidate adds coverage, the order reverts to bi-encoder ranking.
• Training data. Labels come from the CRUX release of Researchy Questions. A Llama judge rates document-subquestion pairs from 0 to 5, and ratings of 4 or higher count as coverage. The bi-encoder trains on 1,929 examples from 847 queries. The estimator trains on 850,000 pairs from 3,000 queries, where the majority-label baseline is about 69%.
• Self-feedback loop. The generator reruns over the unchanged context, asking for about 18 sentences instead of 12. It uses no gold data and no new retrieval.

Evidence

Table 1 ablation, α-nDCG@10 / StRecall@10, TREC RAG 2024:

• Relevance bi-encoder: 0.625 / 0.744
• Plus coverage training: 0.656 / 0.762
• Plus estimator and greedy selection: 0.702 / 0.770
• Gold nuggets: 0.718 / 0.781
• Oracle greedy: 0.887 / 0.858

Coverage training alone accounts for 0.031 of the gain. Estimation and selection add 0.046.

A larger bi-encoder training set did worse: 44,558 examples over 20,000 queries, with stricter positives, scored 0.598 against 0.656. The authors' explanation is that with an empty covered set, the strict threshold discards moderately relevant documents.

Length is not the source of the loop's gain. Asking the first pass for 18 sentences changes transfer rate by +0.000 and answer StRecall by −0.003.

Generation settings were Qwen2.5-7B-Instruct with greedy decoding, a 700-token limit and 250-word document truncation. Llama-3.3-70B-Instruct was the judge. Decomposers compared were Qwen2.5-7B, Llama-3.3-70B, GPT-4o and a document-grounded variant.

Limits

• Small evaluation sets. TREC RAG 2024 has 56 queries and CRUX-DUC04 has 50. Downstream results use 54 TREC queries, which inflates figures by 56/54 against Table 1.
• Judge-dependent labels. TREC document-nugget links are LLM-judged, at about 69% precision and 90% recall against human assessment. CRUX-DUC04 subquestions and links are fully LLM-generated.
• Single setup. Only one generator (7B), one pool (BM25 top-100) and k=10 were tested.
• Truncated page. The fetched text cuts off in the self-feedback section. Its method details, the decomposer and τ sensitivity results, the CRUX-DUC04 numbers and the faithfulness figures are not available here.
• No code or weights. The page does not mention code, weights or a licence.