Build, Don't Play: Self-Improving LLM Research Agents Develop a Hearthstone Battlegrounds AI That Rivals Top Players

What it claims

AutoTavern is a team of six LLM research agents that took Hearthstone Battlegrounds from raw game data to a ladder-level player in 20 days. The paper was posted in October 2026 by Changlian Ma and Limin Wang (Nanjing University, Shanghai AI Laboratory). The headline is the contrast in the title. The agents built TavernAgent, an 18M-parameter transformer trained with PPO, which decides in 0.03 s. The same class of frontier LLMs, asked to play directly, takes up to 26 s per decision and keeps at most 26% of play tasks within the one-second budget a live turn allows. TavernAgent keeps 90%.

How it works

Agents never talk to each other directly. They read and write a typed, versioned research memory with five stores:

• a ledger of numbered findings, where a correction is a new entry pointing back and nothing is edited away;
• a registry of rule decisions and open questions;
• a hand-over note;
• an experiment registry;
• a lessons store loaded at the start of every session.

The roles are research lead, environment, training, opponent modelling, evaluation and audit, and deployment. Short-lived sub-agents run probes.

Five gates stand between any claim and the memory:

• G1 build: the full test suite passes.
• G2 contract: each model is tied to the simulator build it was trained on, and the controller refuses a mismatched model.
• G3 comparison: models are compared only within one build, over several seeds.
• G4 delivery: a candidate is delivered only after a re-check on unseen seeds.
• G5 reality: delivered models play the live ladder, and those games are replayed in the simulator.

Errors the agents trace to their own method become lessons or new gates. G4 and part of G2 came from this process.

flowchart LR
  M[Typed research memory] --> D[Diagnose gap]
  D --> I[Implement + G1]
  I --> T[PPO train]
  T --> E[Evaluate G3/G4]
  E -->|not better| M
  E -->|deliver| L[Live ladder G5]
  L --> R[Replay in TavernGym]
  R -->|184 fixes| M

The loop runs at three time scales: training (minutes), research (hours) and the ladder (days). A scheduler runs registered experiments within a resource budget, so the loop continues without a human present.

Evidence

Scale of the run. About 10,000 experiments, 23 delivered versions and 63k lines of Rust engine code. The run produced 375 rule verification records and 184 fixes from live games. The human sent 418 messages, which is 0.6% of all messages.

Simulator. TavernGym covers all 978 game objects of patch 36.6.3. It is deterministic and runs 190 to 280 complete eight-player games per second on one core.

Simulated progress. Against reality-calibrated opponents, first-place rate rose from 1.9% (first version) to 41.5% (last version). On a harder exam the last version reached 23.8%. Each point uses one simulator build, three unseen seeds and 24,576 games. Random legal play and a scripted bot score 0%.

Live ladder. Rating rose from 6,000 to about 15,000, which is within the server top 100. Over the last 20 records, average placement was 1.60 with 80% first places. The server's ten highest-rated players averaged 2.34 and 59%.

TavernBench. The benchmark has 960 tasks in 40 scenarios across eight categories, with labels produced by the simulator. LLMs tested include GPT-6 Astra, Claude Opus 5.5, Gemini 3.8 Flash, Grok 4.7, DeepSeek-V4.1-Flash, Kimi-K2.6, GLM-5.3 and Intern-S2-Preview. None exceeds 63% on play tasks, against 91% for TavernAgent.

How to use it

The authors say they will open-source TavernBench and TavernGym at https://github.com/MCG-NJU/AutoTavern. No release of TavernAgent weights or the AutoTavern agent framework is stated.

Limits

• Human in the loop. The human sets goals, rules on mechanics the game data leave ambiguous, and physically presses the keys the AI names in the live client.
• Small live sample. The comparison with top players rests on 20 games.
• Single patch. The simulator is tied to patch 36.6.3. The game's content changes with every patch.
• Missing details. The fetched text ends partway through Section 4.2. Compute cost, the six catalogued errors (Section 8) and the full benchmark table (Table 4) are not covered here.