Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

What it claims

Multiverse Computing compressed GPT-OSS 120B to 60B parameters, recovered it in bfloat16, then re-quantized it to MXFP4 using Quantization-Aware Healing (QAH). The 4-bit result beats its own bf16 checkpoint on 7 of 9 benchmarks. It trails on only MMLU-Pro and SciCode, each by under 1.5 points. The largest gains are on long-context reasoning (+7.4 on AA-LCR) and math (+5.6 on AIME 2025). Against the original 120B teacher, it scores higher on LiveCodeBench (66.5 vs 66.0) and comes within 1.6 points on GPQA Diamond (67.4 vs 69.0). It does this with half the parameters and roughly a quarter of the weight memory.

How it works

The method turns on which model serves as the teacher during the quantization stage.

flowchart LR
  A[Original model] --> B[Structural compression]
  B --> C[bf16 recovery]
  C --> D[Quantize to MXFP4]
  D --> E{Teacher for healing?}
  E -->|QAD: recovered bf16| F[Capped at recovered ceiling]
  E -->|QAH: original full-size| G[Second distillation pass]
  A -. precomputed logits .-> G

• Teacher. The teacher is the frozen original full-size, full-precision model, not the recovered bf16 checkpoint. The teacher and student have different architectures, which works because only the output distributions are matched.
• Loss. Training uses KL divergence on logits only, with no hard labels. The teacher's logits are precomputed offline.
• Why it beats bf16. The authors argue the quantization stage becomes a second distillation pass. It transfers information the bf16 recovery stage did not have the time or data to transfer.
• Long context. The healing corpus includes documents up to 32k tokens. These fit in a fixed GPU memory budget through a chunked KL loss from the authors' companion distillation paper. The loss computes KL one sequence slice at a time and never materializes the full vocabulary × sequence matrix.

Evidence

QAH vs QAT, matched conditions. The authors quantized a GPT-OSS 9B model to MXFP4 and tracked the average score on MMLU-Pro, LiveCodeBench and GPQA Diamond during training:

The authors attribute QAT's collapse to its cross-entropy loss, which keeps pushing toward hard labels. KL against a frozen teacher stops exerting pressure once the student matches the teacher.

Efficiency. The authors state the following:

• About 4× less weight memory than the bf16 student.
• About half the compute per token of the 120B teacher.
• For model families that ship in bf16 rather than 4-bit, the combined reduction would be closer to 8× less compute per token.

How to use it

The practical difference is checkpoint selection:

• QAT needs early stopping against a held-out signal.
• QAH, once sufficiently trained, can reportedly be served without that risk.

To apply QAH you need the original uncompressed model available to precompute teacher logits. For 32k-token healing you also need the chunked KL loss.

The post names no released weights, licence, code, commands, corpus size or GPU budget. Those details, the healing pipeline and the distributed-training findings are deferred to the paper.

Limits

• Single run. The headline result comes from one compressed model, GPT-OSS 120B → 60B.
• Narrow QAT comparison. The QAT head-to-head uses a different, 9B model and averages only three benchmarks.
• No QAD comparison shown. The argument that QAD is capped by the recovered checkpoint is made conceptually, not shown with numbers.
• Remaining gap to teacher. The largest gap is on AA-LCR. The post does not give the size of that gap and calls it the hardest capacity to recover.
• Benchmarks. The full list of the nine benchmarks and the per-benchmark scores against bf16 are not all given in the text.