Humanize Olympiad Agents: A Multi-Session Workflow for Olympiad Problem Solving with Solve, Review, Wiki, Verify

What it claims

HOA is an open harness (code: https://github.com/humanfia/hoa) built on the earlier Humanize worker–reviewer protocol. It accepts a candidate solution only when every applicable gate passes: independent semantic review, plus an objective verifier where one exists. A domain wiki fills knowledge gaps along the way.

The page gives a few figures the summary does not:

• IChO with Kimi-K3: 97.1%, called gold-level.
• Lean-Eval: 251/304 is 82.6%.
• PutnamBench: 672/672 covers all problems from 1963–2023.
• Leakage: the authors say the 2026 contests have none, because they are the latest competitions.

How it works

flowchart TD
  P[Problem + acceptance plan] --> W[Worker builds artifact]
  W -->|knowledge gap| K[Wiki query]
  K --> W
  W --> F[Frozen artifact]
  F --> R[Independent reviewer]
  R -->|calls| V[Verifier]
  R -->|findings| W
  V -->|diagnostics| W
  R -->|all gates pass| A[Accepted]

• Isolation. The reviewer sees the original problem, the frozen read-only artifact, the acceptance plan and a summary of the round written by the runner. It never sees the worker's reasoning or conversation history. The authors say this reduces anchoring and prevents silent repairs.
• No self-acceptance. The worker can call the verifier but cannot accept its own output. It must address every finding the reviewer reports. The loop ends on reviewer acceptance or when resource limits are hit.
• Verifier (Comparator, from Lean FRO).
• It builds the submission in a sandbox, matches the theorem against the trusted statement, restricts permitted axioms, and checks the proof with the Lean kernel.
• The paper notes that a proof can compile while still using a disallowed axiom, so the axiom restriction matters.
• Spotting reward hacking and misread problems is the reviewer's job, not the verifier's.
• Wikis.
• Mathematics: after a proof is accepted, a curation agent extracts lemma records. Each record holds declaration names, Lean signatures, a description, the source problem, imports, source locations and verification status.
• Chemistry: built before evaluation from textbooks, olympiad problems outside the test set and online references.
• Physics and coding: no wiki, because the authors judge that knowledge to be already present in pretraining.
• Physics formalization (Science Mango pipeline). 1. A blueprint separates physical laws, given values, earlier-part results and the claim to prove. 2. An agent searches Mathlib and PhysLean via LeanExplore and writes typed Lean statements with sorry bodies. 3. A physics reviewer rejects missing laws, omitted figure parameters, tautologies and statements that assume the answer. 4. The statement is then frozen, so the prover cannot weaken it.

Evidence

The fetched text stops at the start of the experimental setup, so there are no per-task tables, budgets or ablations to report. The only stated comparison points are:

• PutnamBench: Aleph Prover with GPT-5.2 reports 668/672. HOA's 672/672 is therefore a gain of 4 problems over that prior result.
• Open model: Kimi-K3 with HOA matches GPT-5.6 Sol + /Goal on IMO and IPhO, and exceeds it on IChO and IOI. The page does not describe that baseline's setup.

Limits

• Missing experiments. Compute, token budgets, round limits, wall-clock cost, ablations of each feedback channel and the grading method for non-Lean tasks are all absent from the fetched text. Check the full paper before relying on the numbers.
• Coding has no trusted verifier. For IOI, the worker and reviewer write their own test cases and timing checks.
• Chemistry has no stated verifier. Acceptance appears to rest on review, and the wiki is curated by agents from web and textbook sources.
• Physics scores depend on agent-written statements. IPhO results are proofs of statements that agents formalized and agents reviewed. The fidelity of those statements to the original problems is an assumption the reader has to accept.
• Leakage-free status is asserted, not shown. The page claims the 2026 contests have no leakage on recency grounds only and gives no further evidence in the fetched text.