
RetroChessBench: A Benchmark for Reconstructing and Proving Chess Histories
RetroChessBench is a 12-task benchmark for retrograde chess: reconstructing legal game histories, exact-length constructions, unique endings and exhaustive answer sets, with references, rubrics and a terminal-reward RL environment. GPT 6 Astra High fails to give the required final answer in 22 of 24 attempts without tools and 18 of 24 with code execution. Mean terminal RL rewards are 14.75% and 27.19%. The authors link the failures to scarce worked derivations in web training data.
evals, research· Added 31 Oct 2026, published 29 Oct 2026
