Skip to content

Back to the catalog

BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning
arxiv.orgpaper

BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning

BrainTRACE is a report-grounded benchmark for evaluating whether vision-language models can trace the evidence structure required for longitudinal brain MRI interpretation. It contains 7,273 scored VQA instances derived from 1,778 longitudinal patients, 7,299 MRI studies, and ~29k co-registered 3D MRI sequence volumes, organized across five levels of clinical reasoning and evidence demands like longitudinal comparison, multi-sequence integration, and volumetric spatial grounding. Evaluation of 20 VLM configurations finds that current models identify isolated visual cues but rarely compose them into grounded longitudinal clinical interpretations.

TL;DR A new benchmark testing whether VLMs can chain longitudinal, multimodal, and volumetric evidence in brain MRI reports — and current models mostly can't.
Open original ↗
Document

What it claims

The abstract reports a "longitudinal evidence composition gap" across 20 VLM configurations, which include general, medical, frontier and native-volume models. The best closed-form accuracy on the broad track is 41.9%. The best open-ended clinical pass rate is 19.8%. The best case success rate on full longitudinal cases is 1.4%. In short, models handle single multiple-choice cues far better than they chain evidence across serial studies.

How it works

Source data. Items come from the public MR-RATE corpus. Only patients with at least three MRI studies are kept, and each study must have paired imaging, a radiology report and segmentation-derived measurements. Each study centres on a T1-weighted acquisition. Co-registered T2, FLAIR, SWI or other sequences may also be present. References come from real, anonymized MR-RATE reports. Those reports were structured into clinical information, technique, findings and impression fields by an LLM-assisted procedure inside MR-RATE.

Evidence split. Each instance separates what the model sees from what defines the correct answer:

flowchart LR
  R[Longitudinal record] --> V[Permitted visual evidence + question + format]
  R --> H[Hidden reference: target, required facts, contradictions]
  V --> M[VLM]
  M --> S[Scorer]
  H --> S
  R --> D[Diagnostic labels: level, demand, track]
  D --> S

The hidden reference differs by question type:

  • Closed-form questions: a normalized answer.
  • Open-ended and case questions: a list of required facts plus "clinically material contradictions" that make an answer unacceptable.

Human reviewers checked report support, timepoint alignment, visual evidence availability, answer format and redaction.

Labelling. Every item carries two labels:

  • Reasoning depth, five levels:
  • L1: acquisition and detection
  • L2: single-study burden, location and appearance
  • L3: interval comparison
  • L4: trajectory and treatment response
  • L5: case synthesis
  • Evidence demand tag: longitudinal, multi-sequence, volumetric or clinical reasoning.

Three tracks:

  • Broad compatible: 5,653 items, with evidence rendered as standard images so both open and closed VLMs can run.
  • 3D-evidence: 1,200 items using multi-slice renderings or native-volume input.
  • Case reasoning: 70 cases decomposed into 420 steps. The six steps are anchor localization, early interval change, interval characterization, burden trajectory, latest status and final synthesis. This tells you where along the chain a case failed.

How to use it

The authors release the following:

  • benchmark specification
  • evaluation lists
  • scoring code
  • rubrics
  • an audit-record format

Locations:

The text shown gives no install commands and no licence. Check the repo, and the MR-RATE terms that the data inherits, before using it.

Limits

  • Scope. It covers only longitudinal follow-up patients from MR-RATE. The authors say it excludes non-longitudinal referrals and the full neuroradiology spectrum. The worked example is glioblastoma, so disease mix matters, and the cohort composition is in Appendix B.2.
  • Reference quality. Ground truth depends on report text that went through an LLM-assisted structuring step upstream. It is not independent re-reads of the images.
  • Fetched text is truncated. It stops partway through construction, so the per-model results, the named models, the per-level and per-track breakdowns, and any limitations section were not available here. The three headline numbers above are the only results this note can support.

Organisations named here

  • BrainTRACE

Technologies named here

  • Vision language model

Related in evals