Skip to content

Back to the catalog

OpenAI Says a Secret AI Model Cracked Hundreds of Open Math Problems in One Prompt—Mathematicians Want Receipts
decrypt.coarticle

OpenAI Says a Secret AI Model Cracked Hundreds of Open Math Problems in One Prompt—Mathematicians Want Receipts

Decrypt reports that OpenAI published 722 math manuscripts (grouped into 372 result "families") on GitHub, all generated by an unreleased internal model that OpenAI says largely one-shotted problems from a single prompt per single agent. Only 162 of the 722 papers (about 22%) have a Lean-formalized, machine-checked main result, and OpenAI itself warns unformalized results may contain errors; mathematicians including MIT's Andrew Sutherland and the Institute for Advanced Study criticize the release for omitting prompts, chain-of-thought summaries and per-result compute/time data needed to verify the one-shot claim, contrasting it with Anthropic's fully public, Lean-checked Fermat's Last Theorem proof.

TL;DR OpenAI claims an unreleased model one-shotted 722 math results, but only 22% are Lean-verified and mathematicians say the claims can't be checked without the model, prompts or chain-of-thought.
Document

What it claims

OpenAI's public repository openai/math holds 722 manuscripts, grouped into 372 result "families", all generated by an unreleased internal model. A family can bundle a main theorem with companion arguments, consequences or alternative proofs, so 722 counts manuscripts, not solved problems. OpenAI says it posed roughly 4,000 problems and published only the outputs it judged significant. The 372 families are therefore a filtered subset of about 4,000 attempts, not a hit rate.

Results named on the page:

  • A proof that the chromatic number of the plane is at least 6.
  • A proof of Saxl's Conjecture.
  • The Quasi-Riemann Hypothesis, the one result Abhishek Saha places in the top "game changing" tier.

Saha classes most of the others as "exceptional advances within an existing program" or "surprising breakthroughs".

Evidence

  • Compute: the average result used about three hours of ChatGPT Pro thinking compute, per OpenAI. For contrast, OpenAI's Navier-Stokes claim the previous month used 10,000 coordinating agents for 88 hours. These figures are averages; there are no per-result figures.
  • Attempts: OpenAI says some results may have taken multiple attempts, which qualifies the "one prompt" framing.
  • Reasoning traces: abridged reasoning summaries exist for 10 of the 372 results. No prompts were released.
  • Formal verification: a formalization catalog in the repo lists 162 papers with a Lean-checked main result. OpenAI says it will add more as they are obtained.

How to use it

  • Where it is: the manuscripts and the formalization catalog are in openai/math on GitHub.
  • No way to contribute back: Issues are turned off and the repo has never accepted a pull request. Outside formalizations have no upstream channel. For example, Keith Adler is formalizing the Saxl's Conjecture proof in Lean 4 on his own.
  • Readability: Dmitry Rybin describes the plane-coloring proof as hard to follow ("alien math"). It rests on an unexpected equivalence between arbitrary K-colorings and "weakly measurable" K-colorings.
  • No model access: the model is not available. OpenAI says it is still working to release it responsibly.

Limits

  • What a Lean pass covers: it only shows the proof follows from the statement as written in Lean. It does not show that the Lean statement matches the original problem, or that the result is new or important. Even for the 162 formalized papers, the reader must check the statement translation themselves.
  • Unformalized results: OpenAI states that "some of the unformalized results could have issues". Treat the other 560 manuscripts as unverified.
  • Disclosure gap: on September 29, an Institute for Advanced Study advisory group recommended disclosing, for every result, the model name, prompts, a summarized chain of thought, time taken and compute cost. OpenAI provided only averages and 10 summaries.
  • Reproducibility: without the model or prompts, the single-agent one-shot claim cannot be reproduced. MIT's Andrew Sutherland says to treat it as unverified until it can be.
  • Dissent: Daniel Litt (University of Toronto) argues there is no reason for OpenAI to withhold the answers.
  • Anthropic comparison: Anthropic's proof posted all 13 million Lean lines. However, it formalized Wiles's 1995 theorem rather than claiming new results, so it is not a like-for-like benchmark.

Organisations named here

  • OpenAI
  • MIT
  • Institute for Advanced Study
  • University of Toronto
  • Anthropic
  • Abhishek Saha
  • Andrew Sutherland
  • Andrew Wiles
  • Daniel Litt
  • Dmitry Rybin
  • Fermat's Last Theorem
  • GitHub
  • Keith Adler
  • Quasi-Riemann Hypothesis
  • Scientific American

Technologies named here

  • unreleased internal model
  • Lean
  • GitHub
  • unnamed model used for Navier-Stokes claim
  • unnamed model used for Fermat's Last Theorem proof
  • ChatGPT
  • Navier Stokes

Techniques named here

  • Lean formalization