RAQ-MLA: Role-Aware Quantization for Multi-Head Latent Attention Caches

What it claims

RAQ-MLA is a training-free quantizer for the Multi-Head Latent Attention (MLA) cache. It covers both cached tensors, including the decoupled positional key k_R, which prior MLA cache quantizers (Zhang et al., 2026; Muller et al., 2026) leave in full precision while keeping the latent at 4 or more bits. At about 2.5 bits per element, it stays within 0.5 points of BF16 on reasoning and coding across four models from the GLM, DeepSeek and Moonlight families. The authors are from the Institute of Automation, Chinese Academy of Sciences, and Tencent. The preprint is dated 24 October 2026.

Why the two tensors are treated differently

The analysis on GLM-4.7-Flash explains why existing methods lose accuracy at low bit widths:

• Outliers: k_R has fixed-channel outliers, like a standard Key cache. The latent c_KV spreads its magnitude broadly, like a Value cache.
• Score attribution: Pooled over all 47 layers, the positional term carries 92.5% of the attention-logit attribution. The outlier channels, about a tenth of k_R's 64, carry 63.8% on their own.
• Error at INT4: The latent's 12.3% cache error falls to 1.7% at the attention output through the content key, but stays at 8.3% through the value. k_R's error grows from 9.1% to 14.7%.

The tensor that existing MLA methods exempt from quantization is the one that sets the attention score. k_R is only 11.1% of cached elements (64 of 576), so full-precision k_R also limits how far the average bit width can drop.

How it works

1. Granularity: The latent is quantized per token. k_R is quantized per channel.
2. Transforms: Each tensor is stored in a learned invertible basis, A_c (512×512) and A_R (64×64), fitted separately per layer.
3. Objective: Both bases minimize one token-weighted quadratic error, using a per-token "role metric" M_t. - For the latent, M_t is the value up-projection W_UV, weighted by each token's attention exposure (the sum over queries of squared attention weight). The content-key path is ignored, and attention weights are held at their unquantized values. - For k_R, M_t is the second moment of the positional queries that read token t.
4. Fitting: Alternating minimization on calibration prompts, with no gradients through the model and no fine-tuning.

flowchart LR
  L[Latent c_KV] --> Ac[A_c, per-token quant]
  K[Positional key k_R] --> Ar[A_R, per-channel quant]
  Ac --> F["A_c inverse folded into W_UK, W_UV"]
  Ar --> Q[Compensated on query side]
  F --> S[Attention]
  Q --> S

At decode time, nothing is transformed back. A_c's inverse is folded into W_UK and W_UV. The k_R transform is cancelled by adjusting the uncached positional query. The extra parameters per layer are d_c² + d_R², which is 0.08% of GLM-4.7-Flash's parameters.

Evidence

• Proxy check: At layer 23, the latent role-metric loss tracks the exact attention-output error across 128 random transforms (Pearson r = 0.992, Spearman ρ = 0.964).
• Per-layer error: The fitted A_c gives lower attention-output error than Hadamard and OSCAR at all 47 layers, and this holds on held-out validation prompts.
• Accuracy vs. capacity: On GLM-4.7-Flash, a capacity-versus-accuracy plot compares the method with KVarN, QuaRot INT2 and OSCAR INT2. The fetched text does not give per-baseline numbers.

Limits

• Truncated source: The fetched text ends partway through Section 4.3. Benchmark tables, the full list of four models, the per-tensor bit allocation behind "about 2.5 bits", and any ablations are not visible.
• No serving measurements: The 6.5× figure is theoretical KV-cache capacity. No latency, throughput or kernel measurements appear.
• Simplifying assumptions: The latent objective drops the content-key path and the cross-token error terms. The second rests on near-uncorrelated per-token errors (Appendix Figure 9).
• Setup requirements: Calibration data is required. No code, weights or licence are mentioned.