Skip to content

Back to the catalog

Have We Seen an Acceleration in Discoveries?
metr.orgarticle

Have We Seen an Acceleration in Discoveries?

METR note that looks for slope changes in public discovery metrics, with January 2026 as the candidate breakpoint. It finds a sharp rise in vulnerability disclosures (cURL, Firefox, OpenSSL, Microsoft, NVD, OSV), a likely but hard-to-measure rise in math results (Erdős problems, Smale and Green lists), and no clear gain in optimization series such as nanoGPT, CIFAR-10, Stockfish and Gurobi. Data collection was done by agents and may contain errors.

TL;DR Plots public time series across vulnerabilities, math and optimization to test whether AI has accelerated discovery, finding a sharp rise only in vulnerabilities.
Also seen from another angle🛡️ Security →
Document

What it claims

METR's August 2026 note tests public discovery records for a slope change around January 2026. Vulnerability disclosures rose sharply, but much of the rise in large databases is not credited to AI. Math output rose, but only three named open problems fell to AI. Seven optimization series show no acceleration, which the authors call "somewhat of a puzzle" given LLM-optimization results such as Yuksekgonul et al. (January 2026).

How it works

Agents built the time series and the authors audited them. Each series counts discoveries per period, compares 2026 to 2025 or the longer history, and marks AI attribution where the source records it. The data sits in a repository that the authors plan to keep updating. The page does not give the repository's URL.

Evidence

Vulnerabilities. The AI-credited share of the rise varies widely by project:

  • cURL: 9 CVEs in 2025, 36 by 24 June 2026. 15 of those (42%) are AI-marked, and 14 of the 15 name an AI-security employer rather than a method.
  • cURL severity: Low-severity CVEs went from 7 of 9 to 22 of 36. AI-marked findings are mostly Low (12 of 15), against 10 of 21 for the rest.
  • OpenSSL: 6 CVEs in 2025, 39 by 5 August 2026. 18 are corroborated AI finds and 9 more are AI-affiliated but unverified.
  • Firefox: 210 CVEs in 2025, 342 by 4 August 2026. 37 (11%) are AI-marked.
  • Microsoft: 1,243 CVEs in 2025, 1,927 by 11 August 2026. That annualizes to about 2.5 times the 2025 rate, but only 26 (1.3%) carry an AI marker.
  • NVD and OSV: by early August 2026, the NVD's 2026 count equalled its 2025 total, and OSV's already exceeded its 2025 count.

Higher-severity categories accelerate less. Exploitation grows much more slowly than disclosure. Vulncheck's 1H 2026 report shows CVEs up 45% and KEVs up 10% against the prior six months.

Mathematics. Combinatorics (math.CO) arXiv submissions went from 377 in January 2026 to 743 in July, against a 2025 monthly average of 368. The authors say this is not an index of discoveries.

Three listed problems fell with AI in 2026:

  • Smale's Jacobian conjecture: a counterexample in dimensions three and above, by Alpöge and Claude Fable.
  • Green's Problem 44: resolved in the negative by Liam Price and GPT-5.4 Pro.
  • The sofic half of Green's Problem 100: answered by OpenAI Astra.

Green's list had previously resolved just under two problems a year (13 from 2019 to 2025). Erdős solutions rise in 2024, for no clear reason, and again in 2026 with AI-attributed work. The Open Problems Project (78 problems, 17 resolved, none by AI) has slowed.

Optimization. None of the seven series shows a slope change:

  • nanoGPT: about 700-fold faster training from 2019 to 2026, per Parker Whitfill's estimate. Five AI-attributed contributions supplied a small share.
  • CIFAR-10: Hiverge's AI-set record cut the 2.59-second record by about 23%. A later 1.828-second claim from Fulcrum/Fable is unacknowledged and has specification-gaming caveats.
  • Gurobi: releases from 2022 to 2025 gained 13%, 8.6%, 13.1% and 0.6%. That is about 1.4 times cumulative since version 9.5, and slower than earlier eras.
  • Stockfish: about 675 Elo from 2013 to 2026. The only LLM-credited commit is a 0.6% speed patch from July 2026.
  • Matrix-multiplication exponent ω: six post-2010 improvements moved it from 2.3737 to 2.371339, all by human proof.
  • Hutter Prize and MIPLIB: no rise. The Hutter leaderboard has not moved since October 2023.

Proposed explanations. The authors offer four candidate reasons for the uneven gains:

  1. Uneven inference spending. For example, Anthropic donated up to $100M in Mythos Preview credits for vulnerability work.
  2. Domain-specific LLM ability.
  3. Undisclosed progress inside labs. The Fable 5 system card says Claude is limited on frontier-LLM-development requests.
  4. Uneven data quality.

Limits

  • Data quality. Agents collected the data, and the authors expect errors to remain.
  • Public record only. Internal lab discoveries are invisible to this method.
  • Erdős dating. Dates come from paper years, against Thomas Bloom's explicit warning, so the authors treat this as very weak evidence.
  • Microsoft dating. CVEs are batched by Patch Tuesday and dated at fix, not at discovery.
  • Exploitation lag. Exploitation can trail disclosure by months or years, so the KEV gap may close.
  • Gurobi timing. Gurobi releases yearly, so a 2026 jump would not show yet.
  • Bound-tightening series. Data for sphere packing and number-theory exponents was collected but not analyzed.
  • Expert input. Claims of discovery "without expert prompting" can hide expertise spent on choosing problems, training and verifying results.

The authors plan a follow-up on why gains differ by domain.

Organisations named here

  • METR
  • Anthropic
  • Google DeepMind
  • OpenAI
  • Microsoft
  • Google
  • Cybersecurity and Infrastructure Security Agency
  • Clay Mathematics Institute
  • arXiv
  • Firefox
  • CISA
  • Millennium Prize Problems
  • AlphaEvolve
  • Ben Green
  • CIFAR-10
  • cURL
  • Dwarkesh Patel
  • Erdős problems
  • Fulcrum
  • Greg Burnham
  • Gurobi
  • Hilbert's problems
  • Hiverge
  • Hutter Prize
  • MIPLIB
  • Mozilla Firefox
  • nanoGPT speedrun
  • NVD
  • Open Source Vulnerability (OSV) database
  • OpenSSL
  • OSV
  • Smale's problems
  • Stockfish
  • Terence Tao
  • Thomas Bloom
  • US National Vulnerability Database
  • Vulncheck

Technologies named here

  • Claude Code
  • Claude
  • Claude Fable 5
  • Mythos Preview
  • GPT-5.6 Sol
  • AlphaEvolve
  • LLM
  • Claude Mythos
  • GPT-6 Astra
  • curl
  • CIFAR-10 speedrun
  • erdosproblems.com
  • Erdős Problem tracker
  • GPT-5.4 Pro
  • Gurobi
  • Hiverge engine
  • MIPLIB
  • nanoGPT
  • nanoGPT speedrun
  • OpenAI Astra
  • Stockfish

Techniques named here

  • RLVR
  • LLM-driven optimization
  • Slope-change analysis of discovery time series