DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

What it claims

DualQuant, from d-Matrix, learns a per-row scale α and a per-column scale β for each weight matrix by directly minimizing weight-reconstruction MSE. The column scale β then doubles as a SmoothQuant-style per-channel activation rescaling, but it is derived from the weights alone, with no calibration data. The method works for any format because it treats the quantizer as a black box and projects the scales onto each format's allowed scale set:

The authors call it the first dual-scaling method that formally supports the MX and NV formats. Their calibration-free baselines are QuaRot and SINQ+, and they report the largest margins over them on the MX and NV block formats.

How it works

The weight is approximated as W ≈ Diag(α)⁻¹ · Q(Diag(α) W Diag(β)) · Diag(β)⁻¹. At inference the activation is quantized as X·Diag(β)⁻¹, so β reshapes the activation's per-channel statistics. The intuition is that large-norm weight columns coincide with activation-outlier channels, so the β that minimizes weight MSE attenuates those channels.

The objective is non-differentiable because grid assignment is piecewise constant. The solver handles this in two steps:

1. Variable splitting. Each scale is split into an inner copy, which controls grid assignment, and an outer copy, which controls reconstruction. With the inner copies fixed, the problem decouples into independent scalar problems.
2. Block coordinate descent. Each outer update is closed-form: the scale is the energy of the rescaled quantized slice divided by its inner product with the original weight slice. Each update is followed by projection onto the format's scale set and re-quantization.

flowchart LR
  A[Quantize W with α,β] --> B[Closed-form α update]
  B --> C[Project α to format set]
  C --> D[Re-quantize]
  D --> E[Closed-form β update]
  E --> F[Project β to format set]
  F -->|next iteration| A

The method needs no inner loop, no Hessian and no activations.

Evidence

• Toy weight tile (Figure 1). Taking single-scale quantization as 1.00×, SINQ reduces MSE 3.32× and DualQuant 12.54×. SINQ reaches lower matrix imbalance (1.65 vs 4.49), which the authors use to argue that SINQ's imbalance proxy does not track reconstruction error.
• Packed W4A4 run, Llama-3.1-8B on an RTX 4090:
• Perplexity on WikiText-2 is within 0.01 of fake quantization.
• Memory falls 2.51–2.61× and end-to-end speed improves 1.39–2.46× over BF16.
• Fusing β into the activation-quantization kernel avoids extra kernel launches and keeps quantization overhead at 4.3–6.1% of GPU time.
• Accuracy against SmoothQuant. DualQuant matches or exceeds SmoothQuant on WikiText-2, C4 and zero-shot tasks for "most" Llama and Qwen models and formats. The per-model tables are not in the fetched text.

How to use it

• The paper says an "anonymized codebase" exists, but the fetched text gives no link or install instructions.
• The inputs are only the static weights, a quantizer for the target format, its admissible scale set, an iteration count T and an initialization.
• At inference, β must be applied inside your activation-quantization kernel, and the quantized weights are stored with α and β.

Limits

• Not a clean sweep. The authors say DualQuant beats SmoothQuant on "most" models and formats, so check the per-format tables for your model.
• Conditional convergence guarantee. The descent and stationarity guarantee holds only after the quantized grid assignment stabilizes, and it assumes no weight row quantizes entirely to zero.
• β is not a proxy for activation statistics. The authors note that β's correlation with activation statistics is heterogeneous and varies by projection; its benefit is shown causally in their Appendix E.
• Narrow hardware validation. Real-kernel speed was measured only on one model (Llama-3.1-8B) and one GPU (RTX 4090). Throughput on Blackwell, which has native FP4 support, is motivated in the paper but not shown in the fetched text.
• Truncated page. The fetched text stops partway through Section 4, so the full results, ablations and the stated iteration counts were not available for this summary.