
StateBench: Testing how language models track state changes over long documents
StateBench is a benchmark from Allen AI for long-context state tracking, with four difficulty levels (r-trivial, aperiodic, soluble, general finite) grounded in KrohnβRhodes theory. Frontier models (Gemini 3.8 Flash, Claude Opus 5.5, GPT-6 Sol and Luna) fall to 13% or lower on the longest general finite problems. Gated DeltaNet and hybrid models trained directly on the task generalize far beyond training length; transformers never learn the hardest level.
evals, researchΒ· Published and added 3 Nov 2026


















































