Debating RSI, the US-China Gap, and Jaggedness with JS Denain of Epoch AI

What it claims

This is a conversation, not a study. Both speakers say they are highly uncertain. These are the positions an engineer could act on:

• RSI evidence: Denain does not read the public OpenAI and Anthropic acceleration posts as strong evidence of imminent self-sustaining acceleration.
• US-China gap: Denain estimates 6–8 months, measured release to release, and later widens it to "four to eight months seems reasonable." Lambert thinks Kimi K3 and GLM 5.2 were closer to 2–4 months. Denain calls two months "a bit more overfitting-y."
• Distillation: Denain has updated toward distillation being a major factor. Lambert is unconvinced and attributes more of the gap to compute differences.
• Robotics: Lambert expects robotics rollout to follow self-driving cars, not LLMs, on a timescale of decades rather than 2–5 years. Denain thinks factory build-out would not be the bottleneck if robot capability arrived, citing the "elasticity of supply."

Evidence

OpenAI's acceleration post, as described by Denain

• Researchers' Codex spending grew about 2X a month.
• Codex spending elsewhere in the company surged in spring and then plateaued over the summer.
• The uplift shows up in engineering and troubleshooting tasks.
• OpenAI reports little uplift on compute allocation or choosing research directions.
• Denain flags a possible measurement artifact: researchers used ChatGPT before the Codex numbers were tracked.

Contamination check on the gap

• Epoch internally recomputed the ECI lag using only private benchmarks, or benchmarks published after a model's release.
• The overfitting effect on the gap was "basically not statistically significant."
• Denain expects contamination effects would appear if the analysis relied on model-card numbers instead.
• Denain suspects ECI may still understate the gap, because US labs optimize capabilities that benchmarks capture poorly.
• Denain thinks the Artificial Analysis Index may understate the gap, but hedges because its methodology changes.

Counterfactual on distillation

• If distillation were effectively stopped, Denain projects the gap moving from about 6 to about 8–9 months over the following 6 months.

Why Denain updated toward distillation

• The Stolen Thoughts paper showed reasoning traces were relatively easy to extract.
• "Claude routers" supply realistic prompt distributions.
• Claims that mid-training does "80% of the way" suggest that SFT on the trajectories is valuable.

Gap explanations beyond distillation that were discussed

• Data vendors may resell RL environments that were validated with frontier-lab R&D compute. Lambert relays a multi-hop rumor from China of buying "the RL environments that Anthropic bought for a 10th of the price."
• Chinese labs historically released faster after training.
• Proof-of-concept ("four-minute mile") effects.
• Architecture ideas leak quickly through staff turnover.
• Chinese labs may focus more on benchmarks, partly because some are public companies.

Epoch's Chinese job-postings study (Wu & Denain, published June)

• Data-center hiring suggests some labs are building their own data centers rather than renting from Alibaba.
• Z.ai/Zhipu posts far more B2B sales roles than MiniMax or Moonshot.
• MiniMax is pushing harder internationally.
• Hiring is heavily in-house, with many intern roles, possibly for data work.
• Denain says LLMs are good at classifying postings into research, engineering, or go-to-market roles. LLMs are poor at picking which findings are interesting.

Research-automation pattern

• METR's nanoGPT/autoresearch comparisons fit Tom Cunningham's "apple-picking" model: AI is much more efficient early on but uncovers fewer new ideas.

Safeguards

• Denain thinks the near-term open-model risk comes mainly from inference providers serving models without classifiers. Fine-tuning safety away matters less for now, because few people have both the motive and the skill.
• Denain distinguishes synchronous monitoring, which is weak against fast attacks, from asynchronous trajectory analysis, which is better against long campaigns.
• Lambert's view is that any served API should be assumed to have extractable capabilities.

Proposed test of acceleration

• Will Anthropic's best-model ECI slope over the next year be 5X its 2025 slope?
• Lambert would not be surprised by such a jump, but expects it to fade once measurable targets saturate.

Limits

• Truncation: The fetched text stops at about 60k of 66k characters, partway through the "How Epoch AI ticks" segment. The chapter "What a frontier post-training recipe looks like" (1:00:55) is not covered here.
• Source quality: Most figures are verbal estimates or second-hand descriptions of other posts, not Epoch's published methodology. The gap numbers come with explicit noise from the choice of lag-computation method (forward vs. backward, staircase vs. interpolation).
• Hedged claims: The router, RL-environment resale, and safeguard-incident points are presented as gossip or rumor. Denain says Epoch claims no robotics expertise.