🔬 An Oscar, Two Asteroids, and the Algorithm in Your sklearn: John Platt on AI for Science

What it claims

Google's ERA turns any problem that can be written as a score function into a search for code that maximizes it. It started as an "auto-Kaggle" system that could take on any Kaggle problem. Platt says the team has used it to solve enough outstanding problems to produce at least ten papers.

The headline case is contrail climate accounting. Contrails reportedly account for about 1% of human-induced warming, and the proposed fix is to have planes drop a flight level or two. The hard part is measuring how much warming was avoided, which is a counterfactual. The team already had a working model for the heat-trapping half. For two years they had no model for the reflected-sunlight half. ERA found a simple model for it that included confounders the team had not considered.

How it works

The interview gives more detail than the summary on the search loop:

• The system keeps a tree of notebook experiments, and branches share their history so different leaves can learn from each other.
• An Upper Confidence Bound rule picks which notebooks to mutate. The rule is optimistic rather than greedy, so even the fifth-best notebook sometimes gets chosen.
• Gemini proposes about ten mutations at a time.

flowchart LR
  T[Notebook tree + shared history] --> U{UCB select}
  U -->|promising, not just best| M[LLM proposes ~10 mutations]
  M --> S[Run & score]
  S --> T

Model capability is the deciding factor. Platt reports a step change between Gemini 2.0 and Gemini 2.5: under 2.0 the approach did not work, and under 2.5 it worked well. He attributes the difference from 1970s-era evolutionary algorithms to the LLM "knowing where to look."

How to use it

• The GitHub repo is an open-source implementation. It ran on Gemini but can be used with any LLM.
• ERA is not available as a Google product.
• The page links the paper and blog but gives no install commands, cost, or compute figures.
• The prerequisite is your own score function. Platt calls writing it the hard part.

Limits

• Predictive, not descriptive. ERA produces predictive models. Checking that they actually describe the phenomenon is left to the scientist. Platt's words: "It's a power tool. It can slice your fingers off."
• Reward hacking. The search will exploit flaws in the metric. Platt's example is Google's contrail-detection Kaggle competition. The winners exploited a half-pixel label error (whether the origin sits at the pixel corner or the pixel centre). That won the $15,000 prize but did nothing to solve contrails. Treat the benchmark score as a target that Goodhart's law applies to.
• Weak models fail. Given the 2.0 to 2.5 step change, results with weaker models may not carry over. The page reports no tests with non-Gemini models.
• Start with baselines. Platt's advice is to fit linear regression or an SVM first, before running an optimizer like ERA.
• No benchmark numbers. This is an interview, so the page gives no quantitative results, baselines, or ablations. Check the linked paper for those.