Skip to content

Back to the catalog

Teaching Everyone to Fish for Tokens
interconnects.aiarticle

Teaching Everyone to Fish for Tokens

Nathan Lambert's interconnects.ai essay argues open-weight AI models function less like Linux and more like installed software versions, while the true open-source analogue is full training recipes like Ai2's Olmo or EleutherAI's Pythia. It examines why Nvidia is bankrolling near-open-source efforts (e.g. Nemotron, reportedly ~$26B) to sustain demand for its chips, why companies like Databricks and 01.ai have exited model training, and predicts open models will fork toward a long-tail niche (on-prem, specialized, efficiency-focused) unless open ecosystems develop their own platform-like financial feedback loops. It also proposes that the "pretraining/post-training" lexicon may split into pretraining, reasoning training, and post-training as base-model training becomes more opaque and post-training/fine-tuning (e.g. via Tinker) dominates community interest.

TL;DR An economics-of-open-AI essay on why Nvidia and Meta subsidize open-weight models, and why open-source training may fork into a long-tail niche as post-training/fine-tuning eclipses full base-model training.
Document

What it claims

Beyond the Linux analogy and the two-futures framing, the essay adds several points an engineer choosing an open model to build on should weigh:

  • Fewer base models. The number of open model builders who release base models, meaning checkpoints before core reasoning training, keeps falling. The author links this to reasoning training becoming as opaque as at-scale pretraining was a few years ago. If your plan depends on starting from a pre-reasoning checkpoint, expect fewer options.
  • Revenue-share licences are spreading. Open model builders are experimenting with licences that take a share of revenue from downstream product use or inference. The author frames these as experiments to keep near-frontier open-weight training financially viable, and says Nvidia's demand-growth strategy depends on them working.
  • Fine-tuning is where community activity is. Practitioners take models such as DeepSeek V4 Flash, Inkling Small or GLM 5.X and fine-tune them for specific agentic tasks. Tinker is named as the most popular fine-tuning API today.
  • Weights have long shelf lives. Many companies still run workflows on Llama 3, years after agentic behaviour took off.
  • Two funding strategies, different durability. Nvidia funds near-open-source recipes ("teach everyone to fish for tokens"). For Nemotron it releases all the data it legally can, plus the training code. Meta instead "floods the zone" with open weights, for example by releasing Muse Spark 1.2. The aim is to suppress the token-selling revenue growth of Anthropic and OpenAI. The author treats this as indirect monetisation by hyperscalers with large balance sheets, not as a self-sustaining recipe ecosystem.
  • The bar for self-sustaining open training. Any open model company would need financial feedback loops whose rewards are proportional to the profits of Anthropic's and OpenAI's APIs, sustained over decades.

Limits

This is an opinion essay, not a study. It offers no data beyond the reported ~$26B Nvidia figure, which the author attributes to reports rather than confirming. The author calls the long-tail fork the most likely outcome; that is a judgement, not a demonstrated result.

The essay names revenue-share licences as a trend but does not identify which models carry them or what their terms are. Check the licence of any open-weight model you adopt for downstream revenue or inference obligations. The author also wrote the piece as one of the builders of Ai2's Olmo, so it argues from a stake in the open-recipe side.

Organisations named here

  • NVIDIA
  • Ai2
  • EleutherAI
  • Databricks
  • 01.ai
  • Meta
  • Anthropic
  • OpenAI
  • DeepSeek
  • Nemotron
  • Tinker

Technologies named here

  • Olmo
  • Tinker
  • Pythia
  • Nemotron
  • Llama 3
  • DeepSeek V4 Flash
  • Inkling Small
  • GLM 5.x
  • Muse Spark 1.2

Techniques named here

  • Open-source language model recipe
  • Open weight models
  • Post-training
  • Pretraining, reasoning training, and post-training lexicon
  • Revenue share licenses