Skip to content

Back to the catalog

I expect rapid progress but not towards general superintelligence
interconnects.aiarticle

I expect rapid progress but not towards general superintelligence

Nathan Lambert argues that near-term AI progress will come mostly from automating infrastructure and engineering work (superhuman distributed GPU engineers, end-to-end optimization of training and inference), not from models becoming broadly superhuman researchers. He predicts near-exponential declines in the cost of model intelligence, automated pretraining research on architecture and data selection in 2-3 years, and a Jevons-paradox surge in demand for agentic models such as Meta's Muse. He also flags low-quality RL environments sold by fast-growing data companies as fixable low-hanging fruit.

TL;DR Lambert expects rapid, engineering-driven AI gains (cheaper inference, automated pretraining research, better RL environments) rather than general superintelligence.
Document

What it claims

The core claim is that researchers who expect AI to out-do them soon are extrapolating from the wrong signal. They are seeing fast gains in models' infra and engineering ability. Lambert agrees that models will be "superhuman distributed GPU engineers" within a few years. He argues this makes experimentation easier but will not make the models "dramatically different in nature."

His more grounded name for what others call RSI is "parallelized, AI-assisted language modeling." He says most near-term progress comes from scaling inference-time compute with current tools, not from step-changes in research ability. He does not expect economically valuable superhuman traits outside math and coding. As engineering stops being the bottleneck, good ideas become worth more than good execution, which reverses the current norm of judging researchers by how well they implement and scale.

How it works

The argument rests on verifiability. The training and inference stack has metrics that agents can optimize directly:

  • Training: tokens per second per GPU.
  • Inference: tokens per prompt, FLOPs per token, cost per answer.

Each has established sub-problems and architecture trade-offs, so end-to-end agentic optimization should push inference close to the accelerator's compute ceiling.

He describes two phases:

  1. Next few years: capture the low-hanging efficiency gains on GPUs. He says GPU flexibility matters here because architecture exploration is a major lever for automated research.
  2. Longer term: co-design accelerators and models for further orders-of-magnitude efficiency.

Evidence

This is an opinion piece with one quantitative anchor. Labs have already cut serving cost by roughly 10–30% after announcing a model at a given price point. He expects the cost of intelligence to fall "potentially faster than recent trends." For RL environments, he says several data vendors have crossed $100M or $1B in revenue. Researchers widely call much of what they buy "crap," yet leading labs still see clear ROI from buying it. The piece gives no fix for the quality problem beyond calling it "clearly fixable."

Limits

  • Science: he expects models to find many cross-sub-field links in biology and chemistry by connecting sparse literatures. He leaves open whether this is a new era of discovery or just an acceleration of the arc science was already on.
  • Automated pretraining research: the 2–3 year prediction covers only architecture and data selection for the current class of models.
  • Agent demand: the Jevons-paradox claim rests on one example, Meta's Muse. The value he describes comes from orienting and delivering agents, not from frontier performance.
  • Source: the fetched text ends mid-argument at the RL-environment section, so any concrete remedies he gives later are not captured here.

Organisations named here

  • Meta
  • Interconnects
  • Ilya Sutskever
  • Nathan Lambert

Technologies named here

  • Muse
  • GPU

Techniques named here

  • Parallelized, AI-assisted language modeling
  • Inference efficiency optimization by AI agents
  • Accelerator and model co-design
  • Automated pretraining research
  • RL environments