What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents

What it claims

Reasoning effort settings in LLMs behave like a credence good. A user picks a level before knowing whether extra reasoning will help. Even a successful run does not show whether the purchased compute was needed. The paper measures the counterfactual effect of each effort level on proprietary and open-weight models across four benchmarks, covering single-call reasoning and long-horizon agentic tasks. It reports three effects. Higher effort can lose tasks that lower effort solved. It overspends on easy tasks, with the top level costing up to 42x the lowest. It also changes how agents behave, beyond how many tokens they use.

Evidence

The trajectory finding is the one the summary does not cover. At higher effort, agents verify their own work more heavily. They also search for solutions more often instead of working through problems directly. Both habits lengthen trajectories without necessarily changing the outcome. This means part of the extra cost at high effort comes from a change in strategy, not from deeper reasoning on the same path.

The 42x cost figure is measured only on the subset of tasks that every effort level solves. It is not an average over all tasks.

The escalation baseline (start at the lowest effort, move up only after a verified failure) matches or improves on fixed top effort across all models and all benchmarks tested. The cost saving holds for most proprietary models, not for all models. The abstract gives no cost result for the open-weight models under escalation.

How to use it

The policy depends on having a verifier that can confirm a failure before escalating. Tasks without a reliable success check cannot use it as described. The Hugging Face page gives this command to load the paper into an agent:

hf papers read 2610.34491

Limits

The page shows only the abstract. It does not name the models, the four benchmarks, the effort levels, the verification method or the per-benchmark numbers. It also does not say what the escalation policy does when the top level fails. No models, datasets, Spaces or collections link to the paper on Hugging Face as of October 2026.