[AINews] LLMs can read & write neural network weights: "Weight Space Programming" cuts Autoresearch out of the RSI loop

What it claims

The lead story is "Weight-Space Programming" from MIT CSAIL (Rikiya Takehi et al., co-authored with Alex Zhang). GPT-6 Astra inspects and edits neural-network weight tensors from natural-language goals, with no gradient descent or training loop. The headline result edits 131,072 weights (about 0.016%) of Qwen3.5-0.8B and raises MATH-500 accuracy from 7.9% to 48.1%. The primary source is rikiyat.github.io/blog/weight-space-programming. Latent Space places the result alongside Karpathy's autoresearch from about eight months earlier. Its view is that the method skips the code, train and eval loop, but it calls the work a variant of mechanistic interpretability that is unlikely to scale soon beyond simple games and RL policies.

Evidence

The reported demos are:

• Identifying the behaviour of small models, including Atari-style policies, from raw weights.
• Patching a failed PPO car-control policy by editing 3.5% of its weights.
• Writing MinAtar Breakout and other small game-playing MLP weights from scratch, beating some independently trained models.
• The Qwen3.5-0.8B MATH-500 edit, reached after 50 attempts, with claimed instruction-following gains of up to 3x.

In the MATH-500 edit, token-limit failures fell from 93% to 33%. The edit mainly suppresses a reasoning-loop attractor and does not add mathematical skill.

One Reddit commenter attempted an independent reproduction on a small transformer checkpoint. Astra first ran numerical weight checks (nonfinite values, collapsed heads, rank), then made reversible edits under a no-harm rule. A naive bias edit of -0.35 cut false positives from 43 to 33 but also cut correct objects from 32 to 27, so it was rejected. A targeted query/key edit on a 102-page dev set raised link recall from 75.69% to 76.03% and precision from 89.09% to 89.49%. It gained 6 correct links and lost none.

Limits

Reddit critics in r/LocalLlama argue that the claims are narrower than "native" weight reading:

• Experiments 1–2: these may succeed because first and last layers look like spatial heatmaps.
• Experiment 3: this may follow known Elhage et al. QK/OV recipes.
• Experiment 7: reading dense intermediate representations failed repeatedly until the model was allowed to build activation probes.
• Go baseline: the 512x6 MLP opponent is called a weak baseline.
• MATH-500 baseline: the 7.9% figure came from looping under sampling settings the model card warns about.

Critics compare the fix to Arditi et al. single-direction ablation and attribute the gain to removing one obstacle. The same intervention reportedly failed on Gemma 3 270M, which lacks the looping pathology. The work ran in a Python sandbox.

Other items with numbers

GDPx (SemiAnalysis). The benchmark has 206 agentic tasks across 11 domains. Opus 5.5 leads at 12%, and Gemini 4 Carbon scores 10.9% at about 4x lower cost. GDPx is not open source. Some Argon cybersecurity runs may be affected by an Antigravity bug.

DeepSeek super-agent harness. The repo is dsh-superagent-prototype. Its reviewer states that it has no cross-session memory and that state, compute and environment separation is mostly unimplemented. It was validated only in a small, trusted internal environment, and security and agent-to-agent communication remain open questions.

ISTA DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder. The build prunes to 192 of 384 experts per layer and has 72.1 GB of transformer weights plus 110.6 GB of Engram weights. LiveCodeBench v6 drops from 89.71 to 88.57, and SWE-bench Verified from 81.20 to 77.00. It runs in llama.cpp b11870+ and Strata. One commenter reports that another group failed to match ISTA's earlier numbers.

Strata patch. The PhialsBasement/Strata branch dsv41-kv-splice cuts DeepSeek V4.1 agent session startup from about 100 s to about 5 s. The test machine is an RX 9070 XT 16 GB with 64 GB DDR4.

Local quants vs Claude. On 9 private C++/CUDA bugs, judge scores were:

A 3-bit Qwen3.8-27B used 1.40x more tokens per task. Opus 5.5 also served as the judge, which commenters flag as possible bias.

"Numbat" leak. The alleged Anthropic leak is likely fake, because parts of the weights exactly match random initialization.