Intervention-Certified Amplification: A Validity Gate for $σ$-Weighted Post-Training

What it claims

Per-sample σ-weighted SFT/RL hybrids can amplify a pretrained surface behavior that the training data does not contain. The authors' probe is DIAA, trained on Qwen2.5-1.5B base. Its anchor has every literal Python fence removed, yet it emits unexecuted Tool-Integrated Reasoning (TIR) on 29% of MATH-500 prompts. Vanilla GRPO emits it on 9.2%. Pure SFT on a DeepSeek-Math-7B-Instruct teacher anchor with 0% TIR still yields 45.8% student TIR. The authors read this as reactivation of a pretrained channel, not imitation, so data scrubbing alone cannot fix it. ICA multiplies σ by a gate derived from paired-rollout interventions, which cuts TIR and raises accuracy.

How it works

DIAA's loss is σ(x)·L_SFT + (1−σ(x))·L_RL, with σ(x) = sigmoid(−β·Δ(x)). Δ is the policy's log-prob of the anchor trace minus that of an on-policy rollout, so no separate discriminator network is trained. The anchor is self-generated rejection-sampling data: 94,865 GSM8K+MATH train samples filtered by boxed-answer match. Before cleaning, 26.5% of the reward-1.0 traces contained fake python/output blocks.

Drag is measured by running the same checkpoint under a baseline suffix and a positive-framing intervention suffix. The certificate is the accuracy gain on prompts where the intervention removes TIR, minus the gain on never-TIR prompts. ICA sets σ_eff = σ·c(x; θ_c), and c stays at 1 unless the certificate supports suppression. It comes in two forms:

• Frozen: c is computed once from a prior checkpoint's probe.
• Online: θ_c is updated every K_c steps on a slow timescale.

Counting & Probability is left ungated by a subject fallback, because there TIR encodes real math.comb use and the certificate flips sign.

flowchart TD
    A[Sample minibatch, roll out base suffix] --> B{Step mod K_c = 0?}
    B -- no --> E[σ_eff = σ · c]
    B -- yes --> C[Probe rollout, intervention suffix]
    C --> D[Paired Δacc on TIR prompts updates θ_c]
    D --> E
    E --> F[Update policy on σ_eff loss]
    F --> A

The added cost is one probe minibatch every K_c steps and no extra model.

Evidence

All results are on MATH-500 at step 1200. Training uses Qwen2.5-1.5B with LoRA, following TINA's recipe.

The shuffled-certificate and length controls both lower TIR but lose accuracy versus ungated DIAA. The authors take this as evidence that the certificate content matters, not just downweighting. Hard-filtering flagged samples is worse than downweighting them.

Drag splits into comparable parts: SFT imitation contributes +4.7 pp and σ-amplification +5.3 pp. Interventions on the σ branch alone therefore recover only part of the drag. Truly-per-sample σ shows 1.5 to 1.7 times more intervention persistence than per-prompt σ. Anti-TIR prompting backfired, raising TIR by 13.8 pp.

GSM8K test (n=1319) uses gates frozen from MATH-500 with no retuning:

• Ungated: 67.9% accuracy, 24.7% TIR.
• Frozen ICA: 70.7% accuracy, 9.3% TIR.
• Online ICA: 70.4% accuracy, 9.8% TIR.

Limits

• The evidence covers one feature (TIR) in one model family at 1.5B scale. Cross-feature evaluation is future work.
• The MATH-500 certificate is computed on the same eval set as the headline metric. Three pre-registered choices bound the leakage, and GSM8K is the only clean held-out check.
• Two repeat runs preserved the ranking. The authors call these consistency checks, not variance estimates, and draw no conclusion about frozen versus online.
• The "weak win" criterion is a 1 pp accuracy gain. Results remain suffix-dependent.
• TIR is detected by the fence marker, not by whether the code is executable.
• The page states no code, weights or licence.