Large-Scale Biomedical RAG: Corpus Scaling, Document Representation, and Efficiency Trade-offs

What it claims

The HiTZ Center (University of the Basque Country) study scales biomedical RAG corpora from about 300 documents to 4.3 million PubMed articles (117.4M full-text chunks) while keeping gold sources fixed. It reports that findings from small collections often fail to hold at larger scales. Reranking helps only in some corpus and retrieval settings. Abstract retrieval wins when reference evidence is abstract-aligned, and full text wins when richer evidence is needed. TreeRAG-style hierarchical indexing matches or beats flat full-text in most end-to-end comparisons. All generators are 4B-parameter models, so the gains apply to small models.

How it works

• Corpus scaling. Non-gold documents are added at five shared factors (control, tiny, small, medium, large: 1×, 6.6×, 44.1×, 292.6×, 1942.2×), plus a PubMed-only "full" setting of 4.3M documents. MIRIAD at "large" is 1.9M passages.
• Representations. Abstracts are one unit per document. Flat full text is split with a recursive character splitter into chunks of up to 1,600 characters with 200-character overlap. Hierarchical units are built with TreeRAG. MIRIAD has no abstracts, so LLM-generated pseudo-abstracts stand in, and passages of 300 tokens or fewer are kept as they are.
• Index sizes at PubMed full. Abstracts give 4.3M units, flat chunks 117.4M, and TreeRAG 38.3M. On MIRIAD, TreeRAG and flat chunks are nearly the same size (8.0M vs 8.2M at large), so the unit savings come from long PubMed articles.
• Retrievers. BM25, BGE-base-en-v1.5 (about 0.1B) and Qwen3-Embedding-8B.
• Rerankers. Jina-Reranker-v3 (0.6B listwise), BGE-Reranker-v2-M3 (0.6B cross-encoder) and Qwen3-Reranker-8B. Each reranks the top 100 candidates.
• Generators. Gemma-3-4B, MedGemma-1.5-4B, and Qwen3.5-4B with thinking off and on. The generator receives the top 5 units.

Evidence

• Metrics. First-stage retrieval uses Macro Recall@100, reranking uses Recall@5, and end-to-end uses MCQA accuracy. Gold is matched at the source-document level across all representations.
• Conditions. End-to-end runs compare LLM-only, RAG and Oracle. Oracle supplies gold abstracts for MedAESQA and gold passages for MIRIAD.
• Model selection. Configurations were chosen on MedAESQA (40 expert questions) and 200 MIRIAD dev items, then fixed.
• Test sets. Testing covers MMLU-Med, MedQA-US, MedMCQA, PubMedQA and BioASQ yes/no (MIRAGE preprocessed versions), plus 800 held-out MIRIAD items.
• Headline results. RAG improved 3 of 4 generator configurations on the external benchmarks, by up to 9.6 points mean. It improved all 4 on held-out MIRIAD, by 10.2 to 30.3 points.
• Efficiency. Hierarchical indexing used 67% fewer indexed units and 44 to 56% less storage than flat full text. Retrieval latency per question was also measured.

How to use it

The page states that the MIRIAD sampling script is in the authors' main experimental repository, referenced in a Data Availability section. The fetched text does not give the URL. Details of distractor generation, pseudo-abstracts and TreeRAG construction are in Supplementary Sections B to D.

Limits

• MCQA conversion. MedAESQA and MIRIAD are open-ended datasets converted to MCQA with LLM-generated distractors, so accuracy depends on that conversion pipeline.
• Small dev set. MedAESQA has only 40 questions and drives configuration choice.
• Synthetic data. MIRIAD is synthetic, and its abstract-level results rely on pseudo-abstracts.
• Oracle scope. The authors present Oracle results as dataset-specific reference points, not upper bounds.
• Generator size. Only 4B generators are tested.
• Partial text. The fetched text stops at the efficiency methods, so per-benchmark tables, latency figures and per-scale recall curves are not covered here.