Skip to content

Back to the catalog

AI Quanting (@AIQuanting) on X
x.comthread

AI Quanting (@AIQuanting) on X

AI Quanting's X thread covers Moonshot's Kimi K3.1 Pro release (1M-token context, open weights promised by November 4). Moonshot removed cyber data from mid-training and RL stages; on its in-house xdev-bench-hard exploit benchmark (Linux kernel, Node.js, JavaScriptCore, Exim, VirGL bugs) pass@3 fell from 90.5% to 81.0%. The author argues this is only about a tenth of the capability, so four in five expert exploit tasks remain solvable.

TL;DR Thread on Moonshot's Kimi K3.1 Pro, where stripping cyber data from training cut exploit-task success only from 90.5% to 81.0%.
Also seen from another angle🧠 AI →
Document

What it is

Moonshot released Kimi K3.1 Pro, a post-trained version of Kimi K3 with a 1-million-token context window. It is live on Kimi's apps and API. Moonshot says the full weights will be released by November 4. A Highspeed version ships alongside it, which Moonshot says runs 2 to 3 times faster. The page is a thread by AI Quanting summarizing Moonshot's blog post (kimi.ai/blog/kimi-k3-1).

How the cyber mitigation works

Moonshot found cyber content in its general mid-training corpus and in its RL task mix. For the open-weight release it removed that data from both stages and kept ordinary coding tasks. It then scored a no-removal checkpoint and the shipped model on xdev-bench-hard, its in-house benchmark. Most tasks rebuild recently disclosed bugs in the Linux kernel, Node.js, JavaScriptCore, Exim and VirGL. The agent must write a working exploit and run it.

Versionxdev-bench-hard, pass@3
No removal90.5%
Shipped K3.1 Pro81.0%

Moonshot says this shows data removal meaningfully reduces cyber capability. It also says it does not want open weights that extend the absolute frontier of that capability. AI Quanting's reading is that 9.5 points off 90.5 is a little over a tenth, so about four in five of the tasks still fall to a model anyone can download.

Other figures on the page

  • Terminal-Bench 4.0: K3.1 Pro scores 65.7%, against 58.2% for GPT-6 Astra and 66.4% for Claude Opus 5.5.
  • Moonshot's internal hallucination stress test: the claim-level error rate fell from 23.4% for K3 to 8.3%.

Caveats

  • The benchmark is Moonshot's own and is not public in this thread. Task difficulty and the pass@3 protocol are not described.
  • The thread gives no baseline for other models on xdev-bench-hard, so the 81.0% cannot be compared with them.
  • Calling the tasks "expert" and the loss "a tenth" is AI Quanting's framing, not Moonshot's.
  • The weights are promised, not yet released. Any independent check of the 81.0% figure has to wait for the November 4 release.

Organisations named here

  • GPT-6 Astra
  • Kimi K3.1 Pro
  • xdev-bench-hard

Technologies named here

  • Kimi K3.1 Pro
  • Kimi K3.1 Pro Highspeed
  • Linux kernel, Node.js, JavaScriptCore, Exim and VirGL
  • Kimi K3
  • Linux kernel
  • Node.js
  • Exim
  • Terminal-Bench 4.0
  • JavaScriptCore
  • VirGL

Techniques named here

  • Agentic exploit development evaluation

Threat actors named here

  • Moonshot