EpiLens: Lightweight RAG for Epidemiological Knowledge Acquisition

What it claims

EpiLens is a Python package from the ISI Foundation (Turin) for running the same retrieval-augmented analysis over every paper in a corpus. Each output is validated against a pydantic schema and linked to its source passages. The headline workflow, the Paper Classifier, labels papers on four analyst-defined axes: data type, accessibility, geography and paper type. The axes are examples; users configure their own. The case-study corpus has 2,297 papers seeded from the sources of a multi-pathogen parameter study by Ward et al. From that study's early extraction sheets, 327 parameter estimates with published sources were linked to corpus documents. The classifier labels 54% of papers linked to reproduction numbers as model-based, against 22% for fatality risks. The manual labels give 62% and 24%. The gap holds within each of five pathogens. For incubation-period estimates, the model-based share is 55% for SARS-CoV-2 and 6% for mpox.

How it works

A workflow is defined by two artefacts written once. The template holds fixed retrieval queries that set what evidence is gathered for every paper, plus instructions. The schema constrains and validates the output. Parsing, chunking, vector storage, retrieval and schema enforcement run locally under the analyst's control. The LLM is treated as a stateless step: it receives a locally assembled prompt and cannot query the corpus.

flowchart LR
  P[Papers] -->|one-time ingest| V[(Vector index)]
  T[Template queries] --> V
  V -->|top-k chunks| A[Local prompt + schema]
  subgraph Remote or local
    L[LLM]
  end
  A --> L --> O[Validated output + provenance]

Each layer can be swapped independently. Parsing uses unstructured and pypdf, or GROBID for richer section structure. Embeddings come from sentence-transformers or a provider API; the experiments used Gemini embeddings. Storage is a local file index, FAISS or a Qdrant server. Generation goes to OpenAI, Google, Anthropic, OpenRouter or a local model. The package does not depend on LangChain, LlamaIndex or Haystack. The authors trade those frameworks' flexibility for ready-made workflows.

Evidence

The classifier was scored against 214 manually labelled papers. It matched or beat supervised baselines and zero-shot baselines (NLI and prototype classifiers) on all four tasks without task-specific training labels. The zero-shot baselines see only the title and abstract. EpiLens retrieves passages from the body text. The paper says its evaluation shows where body-text retrieval helps most. Per-task metrics are in Appendix D, which is not in the fetched text.

How to use it

Orchestration runs on a laptop. External embedding, generation and vector-database services are optional. Defaults are local-first. GROBID, Qdrant and a Streamlit UI are optional add-ons. Each prompt has a fixed retrieval budget, so per-paper token use stays nearly flat as paper length grows. Cost figures are in Appendix A.3. The paper also describes a fully local deployment and an extraction workflow alongside the classifier.

Limits

The fetched text stops partway through Methods. It gives no per-task accuracy figures, cost numbers, licence, repository URL or install command. The model-based shares from EpiLens run below the manual labels (54% vs 62% for reproduction numbers). The authors say RAG remains vulnerable to low-quality corpora, retrieval errors and context misuse. They do not claim to have solved these problems. They frame the tool as handling mechanical sorting, with judgment left to researchers.