
Does Chain-of-Thought Reliance Follow a Scaling Law?Larger Models Follow Their Steps More, Independent of Post-Training
This paper measures how much a model's final answer depends on its own written chain of thought, using a synthetic box-swap task with exact ground truth. It edits one step in the model's chain to a wrong but valid state and checks whether the answer follows. Across 25 open-weight models, reliance rises with size and saturates (Qwen3 from 16-18% at 0.6B to 97-100% at 14B). Reliance tracks capability (r = 0.99) and is unchanged by post-training.
research, ai-safety, evalsΒ· Added 30 Oct 2026, published 28 Oct 2026


