Skip to content

Back to the catalog

AI Labs’ Call to
galaxy.comarticle

AI Labs’ Call to

A Galaxy Research newsletter piece dissecting Dario Amodei's essay "We Must Pace the Frontier," which proposed embedded third-party evaluators (e.g. METR), cross-lab safety coordination with antitrust waivers, and eventual global AI governance including China. The piece separates Amodei's "operational" (QA/pipeline) and "research" (alignment, interpretability, evals) problems from his "Democratic Coordination" ask, arguing the latter functions as a regulatory moat rather than a necessity, and draws a parallel to Sam Bankman-Fried/FTX-era effective-altruism rhetoric about unilateral existential-risk authority. It covers reactions from Sam Altman (agreeing to adopt evaluators), Trump (rejecting slowdown, citing the AI race with China), David Sacks (questioning METR's neutrality), and Mark Zuckerberg (citing Meta's unilateral delay of Muse and existing liability incentives as sufficient without an industry pact).

TL;DR A crypto-research newsletter critiques Anthropic's "Pace the Frontier" safety-coordination proposal as echoing FTX-era existential-risk rhetoric used to justify a competitive moat.
Document

What it claims

Galaxy Research argues that Amodei's "We Must Pace the Frontier" asks for more coordination than its own examples justify. Its core move is to separate what is urgent from what requires a cartel. The operational and research problems Amodei names can be solved by any single lab acting alone. Only the "Democratic Coordination" step needs competitors and government to agree, through shared standards, speed limits and antitrust waivers. That step is the one the piece reads as a moat. The essay's third step, global coordination including China, explicitly includes pre-release testing and limits on recursive self-improvement (RSI).

Evidence

The piece supports its argument with these specifics:

  • Operational failure. Amodei attributes recent alignment incidents, chiefly the OpenAI/Hugging Face incident, "in part" to "imperfect filtering of broken reinforcement learning environments." Galaxy reads this as a QA and pipeline failure: broken environments should never have reached training. It keeps that separate from the question of why models responded to those environments as they did.
  • Research problems. Alignment, interpretability and evals are treated as methodological problems that labs already compete on. The piece names one concrete eval gap: catching models good enough at concealment to fool existing evals.
  • Unilateral precedents. OpenAI withheld the full GPT-2 in 2019, and Meta delayed Muse. Galaxy uses both to show that "too dangerous to release" needs no industry buy-in.
  • Existing deterrents. Product liability law already covers labs, as it covers all consumer and enterprise software. Commercial and reputational exposure adds a further check.
  • Timing. The essay came within a week of Jacob Coxon's resignation from Anthropic. Coxon had previously worked at OpenAI and wrote that builders believe AI "could kill us all by the end of the decade." Anthropic alignment lead Evan Hubinger amplified that post with ">10% within the next decade." Galaxy says this primed the public for the essay, whether or not the sequencing was intentional.
  • Reaction details. Altman agreed within about 2.5 hours. Sacks's objection was specific: METR is "intertwined with Anthropic's investors and staff," and it would be judging competitors who are not at the frontier.

Limits

  • This is an opinion newsletter, not research. It contains no data, and it quotes Amodei's essay only selectively. Read the original before relying on this characterization.
  • The legal disclosure states that Galaxy and its affiliates may hold financial interests in assets referenced. The firm writes from a crypto and techno-optimist position that it names openly.
  • The piece does not explain the OpenAI/Hugging Face incident beyond the single quoted cause.
  • It concedes that the risk is not zero. It does not argue that alignment or interpretability are solved, only that solving them does not require coordination.

Organisations named here

  • Anthropic
  • OpenAI
  • Meta
  • METR
  • Hugging Face
  • FTX
  • Galaxy Digital Inc.
  • Mark Zuckerberg
  • Sam Altman
  • Sam Bankman-Fried

Technologies named here

  • GPT2
  • Muse
  • Hugging Face

Techniques named here

  • Recursive self-improvement
  • Interpretability
  • Alignment
  • Testing and evaluation

Also named here

  • Dario Amodei
  • David Sacks
  • Jacob Coxon
  • Evan Hubinger
  • Galaxy Digital