Show HN: Gemini 4 Argon plays StarCraft

What it claims

On a fixed 13-bot roster, GPT-6 Astra writes the strongest StarCraft: Brood War bot in one hour. GPT-6.1 Sol is a clear second at just over a tenth of Astra's cost per run, so it is the only model the page rates as good value. Gemini 4 Argon, Claude Opus 5.5 and Claude Fable 5.1 are functionally tied between 37 and 39.

The page's main behavioural finding is that the best predictor of a model's score is how many practice games it plays against the 7 strongest roster bots. The bench also no longer tracks popular aggregate evals. It still tracks Vals IOI and WeirdML v3.

How it works

• Starting point. Each model starts from a skeleton bot that only mines. It also gets the source of the three demo bots and two libraries:
• FAP, a combat simulator
• BWEM, a map analyzer
• Tools. The model can compile its bot, play practice games, and read each game's summary, transcript and its own bot's log.
• Practice tiers:
• D: the three demo bots
• C and B: three mid-ranked human-written bots each
• A: BananaBrain and Locutus
• S: Stardust and PurpleWave
• Final bot. There is no submit tool. Whatever sits in the bot folder at the 60-minute mark is the final bot, so a broken build at time-out is a loss.
• Settings.
• Web search, web fetch and all other internet access are off.
• Every harness compacts context at about 250k tokens. Codex, for example, uses a 272k window and compresses at 90% full.
• Every model runs at max reasoning, except Gemini 4 Argon at "high", the highest it supports.
• CLI versions: Claude Code 2.1.296, Codex 0.162.1, Antigravity CLI 1.3.5, Muse Code 1.4.7.

flowchart TD
    A[Skeleton bot + demo bots + FAP/BWEM] --> B[Edit C++]
    B --> C[Compile]
    C --> D{Practice opponent tier}
    D -->|D/C/B weaker| E[Read summary, transcript, log]
    D -->|A/S strong| E
    E --> B
    C -.->|hour ends| F[Folder collected as final bot]

• Tournament.
• Each model's 5 final bots play all 13 roster bots 6 times on each of three 2-player maps: Heartbreak Ridge, Benzene and Destination. That is 234 games per bot, all Protoss vs Protoss, on fresh seeds.
• Unlike v0.1, the model bots do not play each other.
• A game ends when one side's buildings are all destroyed. Games that hit the 60-minute cap (86,400 frames) go to the higher kills + razings score, as on the BASIL ladder.
• Scoring.
• The score is a plain win rate, replacing v0.1's Elo.
• Because the roster is fixed, a new model can be scored without replaying anyone else, and scores stay comparable across model generations.

Evidence

• Practice behaviour.
• GPT-6 Astra, GPT-6.1 Sol and Gemini 4 Argon start playing the strong bots early in the hour, while their own bots still lose to them.
• Muse Wonder Full, Claude Haiku 5.5 and GPT-6 Luna rarely or never do.
• Claude Fable 5.1 is the outlier: it scores 37 while practising almost entirely against weaker bots.
• Variance across 5 runs. The gap between best and worst run is:
• 18 points for Gemini 4 Argon
• 9 points for GPT-6 Astra
• 8 points for Claude Opus 5.5
• Against the top human bots. Astra wins 7% against Stardust, PurpleWave and BananaBrain.
• New models. Gemini 4 Argon places 3rd and Muse Wonder Full places 7th.
• Correlations.
• Artificial Analysis Intelligence Index: ρ 0.26. Astra only ties for 4th of the 9 models there.
• Terminal-Bench 4.0: the bench has drifted away from it, though it tracked closely in v0.1. Claude Sonnet 5.5 and Muse Wonder Full rank near the top of Terminal-Bench 4.0 but near the bottom here.
• Vals IOI and WeirdML v3: the bench still tracks both. The text does not give the ρ values.

Limits

• Model and harness are confounded. Each model runs in its own lab's CLI, and v0.2 changed both harness and reasoning effort from v0.1. Treat v0.1 and v0.2 results as not directly comparable.
• Reasoning settings are not uniform. Gemini runs at high while every other model runs at max.
• Easier than real tournaments. Only Protoss vs Protoss is tested, only on three 2-player maps. The authors say real tournament conditions are significantly harder.
• Estimated cost. Muse Wonder Full's cost is based on Muse Wonder's API pricing.
• Incomplete correlation data. WeirdML v3 covers only 7 of the 9 models; Gemini 4 Argon and Muse Wonder Full are missing.
• Small samples. Five runs per model, combined with high variance for some models (notably Gemini 4 Argon), limits how far small score gaps can be trusted.