VisionMX: Unlocking Microscaling Post-Training Quantization for Vision Models

What it claims

Direct 4-bit MX conversion is safe for some vision models and catastrophic for others, and the split follows architecture. Under MXFP4-UE8M0 round-to-nearest (RTN):

• Robust: DeiT-B, Swin-T, ViT-S/16 and ConvNeXt-Tiny stay within 5% relative loss with either weight-only or activation-only quantization.
• Severe loss: MobileNetV2 loses 97.40% of its reference accuracy with weight-only and 52.30% with activation-only quantization. EfficientViT-L2 approaches total loss in both modes. YOLOv8-s and DeepLabV3-MobileNetV3 also fall in the high-loss region.
• 8-bit: MXFP8 RTN stays close to float for most models.

VisionMX targets the fragile cases with two components:

• RangeRound, a learned weight rounding that searches a wider set of grid levels.
• Activation Affine Correction (AAC), which is foldable into the next layer.

The authors (Arm AI Research) report gains over PTQ baselines "in most evaluated settings" under joint 4-bit weight and activation quantization.

How it works

Formats. MXFP4 uses E2M1 elements with block scales in one of two formats:

• UE8M0, the OCP power-of-two scale.
• UE5M3, an 8-bit float scale with 3 mantissa bits that gives finer scale steps.

NVFP4 is also evaluated. In weight-only sweeps on ResNet-18 and ConvNeXt-Tiny, UE5M3 gave the highest ImageNet top-1 of the scale formats tested. Scale format alone is therefore a real accuracy lever.

RangeRound. AdaRound-style methods only choose between the two neighbouring grid levels (round down or round up). The paper proves this restriction loses nothing for INT4, given a positive-definite, diagonally dominant Hessian and a unique minimizer. For FP4 the guarantee does not hold, because E2M1 spacings are 0.5, 1 and 2.

Toy example with w = (1.40, 4.90) and off-diagonal Hessian 0.5:

The second assignment has larger weight error but lower loss. RangeRound therefore lets each weight pick from a fixed local range of levels rather than only its two neighbours. The figure shows a 21-point candidate set. Near zero, weights can move across several densely spaced levels; near the clipping boundary, they can round in either direction.

AAC. Post-ReLU activations are nonnegative, so they leave the negative half of the signed FP4 codes unused. AAC shifts and scales activations before quantization and folds the inverse into the following linear layer. This extends asymmetric correction to MX formats.

Evidence

Tasks covered: classification, detection, segmentation and low-light enhancement (RetinexFormer), across convolutional, transformer and hybrid models.

PTQ baselines named in the paper: AdaRound, BRECQ, QDrop and EPTQ. FAAR, a two-level learned rounding method for NVFP4, is cited as the closest MX prior work.

The paper includes ablations on each component and on rounding-range size. However, the fetched text cuts off before the main results tables, so no VisionMX accuracy numbers can be reported here.

Limits

• Head-to-head VisionMX numbers and ablation outcomes are not in the available text. Check the full paper before relying on the size of any gain.
• Improvement is claimed for "most" settings, not all.
• The rounding theorem applies only to the second-order loss approximation under the stated Hessian conditions. The authors note that extending it to the true task loss requires bounding the Taylor remainder.
• The available text mentions no code release or licence.
• If your models are ViTs or ConvNeXt, plain MXFP4 RTN may already be good enough. The method matters most for compact CNNs and hybrids.