Skip to content

Back to the catalog

Breakdown of Local Denoising as Semantic Speciation
arxiv.orgpaper

Breakdown of Local Denoising as Semantic Speciation

This paper studies two temporal phenomena in generative model sampling dynamics: a 'speciation window' where a sample commits to a semantic class, and a 'nonlocality window' where local context becomes insufficient for generation. Under a 'common cause' hypothesis — that semantic labels explain correlations between distant tokens — the authors prove the nonlocality window must fall within the speciation window, and identify conditions under which both windows collapse to a single phase-transition point as system size grows, verified analytically in Gaussian mixture models.

TL;DR Theoretical paper proving that in generative/denoising models, the point where local context breaks down (nonlocality) is contained within the window where samples commit to a semantic class (speciation), with a phase-transition limit verified in Gaussian mixtures.
Open original ↗
Document

What it claims

The page gives three results the summary leaves out.

  • The bound is information-theoretic. It is not tied to one sampler, so the authors say it applies to both autoregressive and diffusion generation, independent of the noising process. Every concrete example and experiment, however, uses diffusion models.
  • The paper proves a probe you can measure. It bounds the "locality gap," Δ_loc, by the conditional mutual information (CMI). Δ_loc is the expected squared difference between the global score ∇_A log p_t(x) and the local score ∇_A log p_t(x_AB). This means the error inside a denoiser can stand in for CMI, which is hard to estimate in high dimensions.
  • The paper motivates its question with outside evidence. It cites prior observations (Zhang et al. 2026) that the two windows nearly coincide in DiT-XL and Stable Diffusion 3. The theory is meant to explain that alignment.

How it works

  • Tripartition. Each image is split into three regions: A is the pixel or patch being denoised, B is a square annulus of radius r around A, and C is the rest of the image.
  • Nonlocality window. This is the time range where I_t(A;C|B) exceeds a tolerance δ. Building on Hu et al. 2025, CMI bounds the error of a local denoiser. When CMI is zero, the score on A depends only on A and B.
  • Why CMI can rise under noise. CMI has no data-processing inequality in the conditioned variable B. Noising B can weaken the A–B and C–B correlations and strengthen A–C dependence, so CMI can grow during noising.
  • Speciation window. This is tracked through I_t(S;R), the mutual information between the label S and a region R.
  • Lemma 1 sandwiches I_t(S;R) between quantities based on the success rate of an ideal forward–backward (FB) experiment: noise region R, resample from the posterior, and check that the label is unchanged.
  • Under exact posterior sampling, P_FB runs from 1 at clean data to Σp(s)², which is 1/L for L balanced classes.
  • So P_FB and the mutual information drop over the same window. Local-region speciation marks the window's start and global speciation marks its end.
  • Common-cause hypothesis. The class label explains part of the dependence between distant regions. Under this hypothesis, CMI is bounded above and below by mutual-information terms, which places the nonlocality window inside the speciation window.
  • Window collapse. The general condition for both windows shrinking to one point is that semantic classes separate faster than within-class fluctuations as system size grows. The paper checks this exactly in Gaussian mixtures as data dimension goes to infinity.

Evidence

  • Theory. Proofs and exact Gaussian-mixture calculations, including the time scales of both windows.
  • Empirical. One ImageNet crop experiment (Appendix B) using a classifier-logit cosine probe. The empirical peak switches quickly, and the switch happens earlier for smaller crops, which fits local speciation preceding global speciation.
  • The fetched text does not include numerical results for that experiment.

Limits

  • Autoregressive models are untested. The authors leave direct tests to future work and point only to a token-entropy-spike result (Zhu et al. 2026) as a possible link.
  • The common-cause hypothesis is assumed. The authors call it "natural for many real datasets" but do not verify it here.
  • The analysis is idealized. It assumes exact posterior sampling and true labels. The authors note that real FB experiments do not reach the ideal endpoints because samplers and classifiers are imperfect.
  • The score-based argument in §3.1 is a heuristic. It rests on a "score-gap suppression" assumption.
  • Window collapse is shown analytically only for Gaussian mixtures.
  • The fetched text cuts off in Section 3.2. Theorem 1's exact statement and the later sections were not visible.

Related in research