Subliminal Learning as Trait-Direction Drift: A Mechanism and Targeted Control under SFT Distillation

What it claims

The paper explains subliminal learning — where a system-prompt-biased teacher generates semantically clean data (e.g., number sequences) that transfers a hidden trait to a student during SFT — as trait-direction drift: biased sampling creates a positive expected preference gap in teacher data, student-recognizable gaps produce trait-aligned SFT updates, and these accumulate into behavioral transfer. Its defense, probe-space corridor regularization, cuts malicious-response transfer from 29.55% to 6.45% with low main-task cost, and suppresses animal-preference transfer in the Qwen setting. It also reports that cross-model transfer is substantially weaker than same-model transfer.

How it works

The mechanism runs in two stages plus a control derived from them:

• Stage 1 (teacher side): Proposition 1 shows the biased-sample expectation of the preference gap G equals an average of KL divergences between biased and neutral prompt distributions, so it is provably ≥0 (strict when the biased prompt changes outputs).
• Stage 2 (student side): Under a local low-rank log-probability factorization, a sample's student-side gap approximately equals its projection onto the trait direction d_trait (the biased-minus-neutral system-prompt embedding). Theorem 1 gives drift δc = S_T + N_T; positive drift is guaranteed when accumulated preference signal S_T exceeds residual/optimizer noise |N_T|. Notably it does not require strong pointwise gap–update correlation.
• Control: A fixed offline probe bank gives a one-dimensional drift surrogate δĉ(θ) = w_probe·(ℓ_θ − ℓ_θ0). Training adds a penalty λ[ReLU(δĉ−τ_high)² + ReLU(τ_low−δĉ)²] — zero inside a calibrated corridor [τ_low, τ_high], penalizing only measured drift outside it.

Evidence

• Low-rank premise: across eight model–trait settings, rank-3 SVD retains ≥99.3% of raw log-prob energy; after removing the neutral baseline, rank 3 retains 72.5–90.3%, rank 5 retains 84.9–95.1%.
• Mechanism ladder (Table 1): predicted vs. measured one-step effects track near-perfectly (e.g., Qwen malicious r=0.182 vs 0.183). The 200-step accumulated drift correlates strongly with initial bin-mean gap: r=0.845 (Qwen malicious), 0.835 (Qwen owl), 0.769/0.805 (Llama).
• Gap-based selection of replacement examples beat uniform sampling in all six model–trait combinations.

Models: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Gemma-3-12B-IT (cross-model matrix); students trained with LoRA. Baselines are base model and standard SFT.

Limits

This is a preprint (v2, Sep 2026); the fetched text cuts off mid-experiments (Fig. 3), so full defense-vs-SFT behavior tables and cross-model numbers are not in view here. Same-model results carry the strongest claims — the paper itself notes cross-model transfer is weaker and depends on how much source signal the target recognizes, and it leaves open whether numerical carriers enter the same persona subspace as prompt-induced traits. d_trait is computed once at θ₀ and held fixed. Carriers beyond number sequences and full setup details are deferred to appendices.