Qwen 3.8 27B on my laptop can interpret model weights, too

What it claims

Simon Willison reports that Qwen 3.8 27B, a 17GB local model, answers the "Which game was it trained on?" task from Sparks of Weight-Space Programming (Takehi, Teoh, Zhang and Khattab) correctly for Pac-Man, Pong and Breakout. It uses no tools and works only from raw policy-network weights given as text. The paper's original claims concern GPT-6 Astra. Willison argues the no-tools question should be judged separately from the paper's code-assisted Qwen3.5-0.8B editing result, which raised MATH-500 from 7.9% to 48.1% by editing 0.016% of weights.

How it works

The paper trained four RL policy networks on simplified 10x10 grid versions of Breakout, Pong, Pac-Man and Freeway. Each prompt holds one network's weights as floating-point numbers, followed by the question. The prompt says there is no code execution, no calculator and a single response. Willison checked that the four prompts are identical apart from the weights. The prompts live in the exp1_which_game folder of the GitHub repository RikiyaT/lm_understand-logs.

Evidence

Each Qwen answer also describes game mechanics, such as a right-side vertical paddle for Pong and a bottom-row paddle with top-row bricks for Breakout. Its answers describe an 8x8 field, while the paper describes 10x10 grids. The reasoning traces count rows, columns and channels by hand. Willison calls them "almost incomprehensible."

How to use it

The prompts are self-contained, so the llm CLI can load them by URL with -f. For OpenRouter, this needs the llm-openrouter plugin:

llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/breakout/prompt.txt \
  -m openrouter/moonshotai/kimi-k3.1-pro

For a local run, load Qwen 3.8 27B in LM Studio and install llm-lmstudio:

LMSTUDIO_TIMEOUT=900 \
  llm -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \
  -m lmstudio/qwen/qwen3.8-27b

LM Studio's default 8192-token context failed with errors, so the model must be loaded with a long context window. The timeout increase is needed because of a plugin issue Willison links. Replace pacman or breakout in the path to run the other games.

Limits

These are single runs per game with no repeats. The Freeway result was still pending when the post was written, and Willison expected it to do worse. The post does not report the laptop's hardware. Willison thinks a skilled human with pencil and paper could reach the same answers with enough effort. He rejects wider claims that the result signals the end of source code. Matthew Honnibal called the paper misleading for mixing the no-tools and code-assisted results. Sean Goedecke argued that the Qwen3.5-0.8B fix follows the same steps a human ML researcher would take.