Covariate-dependent Nonparametric $g$-modeling for regression via infinite Mixture-of-Expertizing class

What it claims

Empirical Bayes regression where each unit has its own coefficient vector βᵢ, drawn from a prior that depends on an observed covariate zᵢ. The prior is estimated nonparametrically by maximum likelihood (NPMLE). The covariate z that drives the prior can differ from the regressors x. For example, z can be latitude and longitude while x holds housing characteristics. The model splits heterogeneity into two parts:

• a part explained by z, which moves the prior;
• an irreducible spread among units with similar z.

How it works

Base model. A unit-level GLM: yᵢ follows an exponential family with ηᵢ = xᵢᵀβᵢ and βᵢ ~ G*(β | zᵢ). Gaussian, Bernoulli, Poisson and gamma families with standard links all satisfy the stated assumptions.

Why the naive estimator fails. Maximizing over all conditional priors is ill-posed. Placing a point mass at each unit's own maximizing β reaches the per-unit likelihood upper bound, so the solution interpolates the data.

The fix: the iMoE class. The prior is restricted to a mixing measure H over pairs (β, γ) on compact sets B × Γ. The conditional prior at z weights each β by e^{γᵀz} and then normalizes. When H is discrete, this is a softmax gate:

gⱼ(z) = αⱼ e^{γⱼᵀz} / Σₖ αₖ e^{γₖᵀz}

The support points βⱼ are shared across all z; only their weights change with z.

Existence and finite size. The log-likelihood can be written as Σ(log Ξᵢ − log Λᵢ), which is linear in H inside a convex hull. Carathéodory's theorem then gives a discrete maximizer with at most 2n support points. That is the bound on the number of experts.

Resulting model. The fitted marginal density is a softmax-gated mixture of experts. z feeds the gate, x feeds the GLM experts, and the gate-weighted sum gives the density:

flowchart LR
  Z[covariate z] --> G[softmax gate g_j]
  X[regressors x] --> E[GLM experts β_j]
  G --> F[weighted sum f y given x,z]
  E --> F

Fitting. The two algorithms extend the exemplar method of Jiang & Guntuboyina (2025), which was developed for a single shared prior in the Gaussian linear model. They use a first-order optimality condition: the directional derivative of the likelihood toward adding a point mass at (β, γ). Experts are selected automatically during optimization. A post-hoc aggregation step then merges the estimate into a few support points with aggregated softmax gates, for interpretation.

How it differs from earlier covariate-aware empirical Bayes. Earlier methods let the covariate set only the prior's mean, or the weight in a two-group mixture. Here the covariate moves the entire nonparametric prior.

Evidence

The fetched text has no quantitative results. Its only applied illustration is qualitative, on the California housing data:

• Response: median house value per census block.
• Regressors: six explanatory variables.
• z: geographic coordinates.
• Finding: the estimated priors for the MedInc and HouseAge coefficients shift visibly along a path through San Diego, Los Angeles, Fresno, San Francisco, Sacramento and Redding.

An NHANES depression-score application and simulation studies are listed, but their results are not in the fetched text.

Limits

• Truncated page. The text stops at the optimality condition in §2.3. The algorithm details, simulations, real-data numbers and discussion are missing.
• No code. The page mentions no code or software.
• Assumptions to check before relying on the theory:
• B and Γ must be compact.
• The exponential-family density must be strictly positive and continuous in β.
• The constant term must be excluded from z.
• Lipschitz continuity of the prior in z holds only for bounded gate parameters.
• Discrete supports. The estimated prior is always discrete with shared support points. Only the weights vary with z; the coefficient values themselves do not.
• Version. This is a v1 preprint (stat.ME), posted 6 Oct 2026.