Quantize by Doing: Task-Guided Mixed-Precision Quantization of Vision-Language-Action Models

What it claims

QbD-VLA picks per-group weight and activation bit widths for a VLA model by running short closed-loop rollouts in simulation. It keeps a lower-precision move only if task success stays above a floor. On LIBERO, uniform round-to-nearest W4A4 gives 0% success on π0.5, OpenVLA-OFT and X-VLA. The selected allocations reach 98.5%, 97.5% and 97.0% on LIBERO-Spatial, against 99.0%, 100.0% and 97.0% at full precision. Nominal widths are 4.00 to 4.16 weight bits and 4.47 to 5.94 activation bits. The authors say the search needs a few hundred simulation episodes.

How it works

Quantization uses fixed round-to-nearest with no calibration and no policy training. The groups are the vision encoder, language backbone, any vision-language projector, and the action head. The action head is split further into attention, feed-forward and modulator groups, following VLAQuantBench. Each group moves down the ladder FP, W8A8, W4A8, W4A6, W4A4.

Cost is parameter-weighted mean weight bits plus mean activation bits. The activation term is not measured activation memory. The floor is τ times the full-precision short-horizon score: τ is 0.75 for π0.5 and 0.90 for the other models. If that score is below 0.5, allocation does not start.

Rollouts use a shortened horizon: 120 steps, or 300 for X-VLA. Each batch has one episode per task, and there are at most three batches per candidate. The search loop:

flowchart TD
  A[Current config] --> B[Next one-group drop, largest cost cut]
  B --> C[Run one batch]
  C --> D{Wilson lower bound >= floor?}
  D -->|yes| E[Accept, re-enable rejected moves]
  E --> A
  D -->|no| F{Upper bound <= floor?}
  F -->|yes| G[Reject]
  G --> B
  F -->|no, batches left| C
  B -->|no moves pass| H[Pick lowest-cost config near best estimate]

Acceptance uses z=1.28 and rejection uses z=1.96. Rejection needs stronger evidence because a wrongly rejected move is never revisited. Results are cached per configuration. Allocation episodes use initial states 20 and up, so they are disjoint from the states 0 to 19 used in final evaluation.

Evidence

The paper reports that component interactions make per-group sensitivity scores unreliable. On π0.5, two action-head groups lose 1.0 and 5.5 points when quantized alone and 97.5 points together. Lowering the π0.5 action-expert MLP from A6 to A4 costs 23.5 points when three other groups are at A6, and 0.5 points when they are at A4.

Other allocators at the same cost score much lower on π0.5: 27.5% when selected by action error, 0.0% by activation kurtosis, and 52.0% by reconstruction error.

LIBERO four-suite averages (Spatial used for selection, applied unchanged to the others):

• π0.5: FP 96.8, uniform W4A6 96.5, QbD W4A5.74 96.2.
• OpenVLA-OFT: FP 96.5, W4A6 22.4, QbD W4A5.94 91.8. On Long it scores 80.5 against 92.5.
• X-VLA: FP 97.6, W4A6 61.9, QbD W4.16A5.90 95.8.
• π0: FP 74.6, W4A6 72.0, QbD W4.14A4.47 73.5.

On π0.5, the released methods were run under the same protocol. HoloQ-VLA at W4A4 averages 98.0. ActQuant at 3.09 weight bits with 16-bit activations averages 95.9. QbD averages 96.2.

On SIMPLER (X-VLA, 864 visual-matching episodes), QbD reaches 66.8% against 64.8% for uniform W4A6.

How to use it

Code, evaluation records and a results video are at github.com/jiuyixu25/QbD-VLA. The search needs a working simulator for the target suite and a policy that scores at least 50% on the short-horizon reference. The page states no licence and no install commands.

Limits

• Real robot: on a Franka Research 3, the frozen X-VLA allocation succeeded on 13 of 30 layouts, against 11 for full precision. The π0.5 allocation succeeded on 0 of 6 layouts, while full precision succeeded on all 6. The authors conclude that simulation success does not guarantee success on hardware.
• Small gains on some models: on π0.5 and π0, uniform W4A6 is already close to full precision, so the gain there is small.
• Stronger baseline exists: on π0.5, HoloQ-VLA scores higher at lower activation precision.
• Nominal bits only: the reported widths count unquantized groups as 16 bits. The fetched text reports no latency or memory measurements.
• Truncated page: the text stops in Section 4.2. Results for π0.5 under uniform settings on the robot, and any further ablations, are not visible.