arxiv.orgpaper

WearableQA: A Benchmark for Health Reasoning over Real-World Wearable Data

WearableQA is a benchmark of 4,084 10-option multiple-choice questions built from real wearable time series, blood biomarkers, and demographics of 200 users tracked for up to 500 days. It defines 16 question types across two axes (data vs. health reasoning, single- vs. cross-signal reasoning) and uses a dual-grounding framework combining literature-grounded and population-grounded physiological patterns to generate reliable questions. Evaluating 14 proprietary and open-source LLMs shows accuracy ranging from 19.6% to 72.9%, with most models below 60%, indicating the benchmark remains far from solved.