Beyond Compression: Training Latent Representations for Stable Long-Horizon Rollout in Neural Surrogate Solvers

What it claims

The summary treats this as three interventions; the page establishes five distinct training-level interventions for sequentially-trained latent dynamics models (LDMs), and locates the novelty precisely: two of them (noise injection, multi-step training) already exist in the literature but were applied to the dynamics model in pixel space (MeshGraphNets, FourCastNet); here they are applied to the autoencoder/compression training to reshape the latent representation itself. The 40% long-rollout error reduction the summary states generically is specifically the number for the active matter problem, one of three case studies (the others: spinodal decomposition, crystal-plasticity fatigue).

How it works

An LDM is an encoder f_ϕ, decoder g_ψ, and a dynamics model l_ζ predicting z_{t+1} from a truncated latent history plus PDE parameters P. Trained sequentially: autoencoder first, purely for reconstruction; then the dynamics model on frozen latent trajectories. The five interventions:

• Spatially structured latent encoding — the only architectural change. Replaces the baseline's flat-vector bottleneck (500:1 compression) with a fully convolutional wavelet autoencoder retaining image-like spatial structure at 64:1, paired with an axial-vision transformer (AViT) dynamics model instead of a plain transformer.
• Koopman-inspired constraint — adds a loss term penalizing reconstruction of the next step via a learned operator K (here a bias-free kernel-3 convolution, not the usual banded linear map), with a ‖K‖ penalty. Weakly weighted 1:2 against primary reconstruction so linear dynamics are encouraged but not at reconstruction's expense. K is discarded after autoencoder training.
• KL divergence regularization toward a unit Gaussian latent.
• Noise injection at two stages.
• Recursive / multi-step rollout training.

The central empirical pattern: interventions that improve long-horizon stability often degrade one-step and reconstruction accuracy — the standard metrics point the wrong way.

Evidence

• ~40% long-rollout error reduction on the active matter problem.
• Matches or exceeds full-resolution models on two physics benchmarks while using 2 orders of magnitude fewer FLOPs and half the GPU memory.
• On mesoscale crystal-plasticity high-cycle-fatigue simulation, the surrogate stays stable extrapolating "orders of magnitude beyond" the training horizon, trained only on computationally accessible data.

Limits

Fetched text is the abstract, intro, related work, and the start of Methods; the interventions after noise injection are cut off, and no results tables, per-intervention ablation numbers, or dataset sizes are visible here. The paper deliberately forgoes architecture optimization, so gains are attributed to training strategy, not tuned models. The framing is confined to sequentially-trained LDMs; joint-training approaches are out of scope.