Skip to content

Back to the catalog

How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?
arxiv.orgpaper

How Does Distribution Shift Shape Pretraining Gains in Neural PDE Surrogates?

This paper studies how distribution shift affects the benefit of pretraining neural PDE surrogates for computational fluid dynamics (CFD). The authors pretrain a surrogate on 254,909 RANS solutions from one airfoil family and fine-tune on a new airfoil family under two settings (same Spalart-Allmaras turbulence modeling vs. added e^N transition modeling), finding that pretraining's sample-efficiency advantage over training from scratch shrinks as target-data budget grows and depends on whether source and target physics match. The results show pretraining value is jointly determined by target-data budget, data coverage (diversity of airfoils sampled), and physics mismatch between source and target domains.

TL;DR Pretraining gains for neural PDE surrogates depend on target data budget, coverage, and physics mismatch, per a study fine-tuning airfoil RANS surrogates across turbulence-modeling shifts.
Open original ↗
Document

What it claims

Pretraining a neural PDE surrogate on airfoil RANS data buys a measurable sample-efficiency advantage when fine-tuning on a new airfoil family, but the size of that advantage — and even its ordering across physics settings — flips depending on the fine-tuning budget. The page quantifies this with specific equivalent-sample multipliers the existing summary leaves out, including a reversal effect that changes what you'd conclude about pretraining's value.

Evidence

Source pretraining: 254,909 RANS solutions from one airfoil family. Fine-tuning on a new family under two targets with matched freestream ranges: same Spalart-Allmaras (SA) turbulence modeling, versus SA plus $e^N$ transition modeling.

  • At N=1000 fine-tuning samples, the pretrained model matches a from-scratch model trained on 3.25× as many samples for the same-SA target, but only 2.58× for the transition-modeled target — i.e., pretraining helps more when physics matches.
  • At N=5000, this ordering reverses: 1.56× (same-SA) versus 1.86× (transition). Both multipliers shrink as budget grows, and the physics-mismatch target retains more relative benefit at the larger budget.
  • On data coverage: at N=1000, sampling more distinct airfoils lowers error on both targets, but the gain exceeds draw-to-draw variation (reported as a 3.3× to 4.0× range) only for the same-SA target. For the transition target, the coverage gain is within noise.

The takeaway the numbers support: pretraining value is jointly set by target-data budget, coverage (airfoil diversity), and physics mismatch — and these interact, so a benefit measured at one budget does not extrapolate to another.

Limits

This is an abstract-only page (588 KB submission, v1, submitted 17 Sep 2026); no PDF internals, architecture, error metric definition, or compute figures are given here. The study is narrow by design: one source airfoil family, one target family, one turbulence model (SA) with one transition add-on ($e^N$), and RANS-only CFD. No claim is made about other geometries, PDE classes, or beyond-RANS fidelity. The coverage finding for the transition target is explicitly within observed variation — treat it as inconclusive, not negative. Before relying on the multipliers, check the underlying error metric and airfoil-sampling protocol in the full paper.

Organisations named here

  • Sai Bhargav Pochinapeddi

Techniques named here

  • Pretraining then fine-tuning for PDE surrogates

Related in research