Rho: A Foundation for Efficiently Adaptable VLA Models

What it claims

Microsoft Research releases Rho, a family of 5B-parameter open-weights VLAs: a foundation checkpoint, Rho-base, plus three embodiment-midtrained variants, Rho-YAM-Box, Rho-UR-AI-Trainer and Rho-FR3-Duo. The headline quantitative claim is that embodiment midtraining halves the task-finetuning data needed to reach a given success-rate threshold, compared with finetuning Rho-base without midtraining. The authors also claim the variants match or outperform existing open-weights VLAs across their tasks and embodiments. The fetched text gives no per-task numbers for either claim.

How it works

The training recipe is called "regularized progressive adaptation". Each stage tries to keep the capabilities gained in the previous one.

1. Backbone (Phi-Phy, 4.68B). A Phi-family causal decoder with 32 layers and width 3,072, plus a SigLIP 2 SO400M NaFlex vision encoder (428M) and a two-layer GELU projector. It is further SFT'd on robotics-relevant VQA. Of the 100M SFT records, about 6M are robotics-relevant: MGrounding, RefSpatial, XVR, RoboPoint subsets, reprocessed multi-image grounding/OCR/counting data, and synthetic chart/UI difference-spotting data. Another 30M are general perception and grounding data.
2. Action expert (542M). A flow-matching transformer with: 12 blocks, width 2,048, MLP width 4,096 GQA with 16 query heads and 4 KV heads at 128 dimensions each * shared adaLN-single timestep conditioning with block-specific zero-initialized offsets

Every block cross-attends to the hidden states of decoder block 14, projected 3,072→2,048. This VLM context is computed once and reused across all 10 Euler integration steps. Robot state enters through a linear projection, outside the VLM sequence.

1. Multi-embodiment pretraining in end-effector-pose action space. The data mix is dominated by dual-arm demonstrations and also includes Web VQA. The backbone trains at a small learning rate, the expert at a larger one, and the token-embedding table stays frozen.
2. Embodiment midtraining. Multi-task training on a single target robot.
3. Offline task finetuning.
4. Optional online adaptation. A latent policy maps each observation to the flow's initial noise instead of sampling it randomly. Only this module learns from corrective feedback; the action expert stays frozen.

flowchart LR
  P[Rho-base] --> Y[Rho-YAM-Box]
  P --> U[Rho-UR-AI-Trainer]
  P --> F[Rho-FR3-Duo]
  Y & U & F --> T[Task finetune]
  T --> N{Initial noise}
  N -->|default| R[Random Gaussian]
  N -->|online adapt| L[Latent policy]
  R & L --> E[Frozen flow expert]

Evidence

• Simulation: RoboTwin, LIBERO and RoboEval.
• Physical robots: YAM Box, UR AI Trainer and FR3 Duo.
• Midtraining: halves the finetuning data needed (see above).
• Online adaptation: works from as few as 15 corrected episodes.
• Ablations: the authors say these show little benefit along the capacity dimensions they trimmed in the expert. They report that the expert's shape matters most for contact-rich and precision tasks on RoboEval.

The tables backing all of this (Section 7) are not in the fetched text.

How to use it

• Code: https://github.com/microsoft/rhobotics
• Weights and data: https://huggingface.co/collections/microsoft/rho
• Project page: https://microsoft.github.io/rhobotics

Interface details relevant to a new robot:

• State and action: continuous vectors of up to 32 dimensions. Joint or end-effector control, delta or absolute targets, and single- or multi-arm setups are all supported. Smaller vectors are padded and loss-masked.
• Cameras: pretraining resizes each view, with padding, to 256×256. Multiple views are interleaved with the instruction.
• Chunking: chunk length and control frequency are training-recipe choices, and the execution horizon is set at deployment. The model is compatible with real-time chunking (RTC) for asynchronous chunk updates.

If your robot is one of the three supported embodiments, start from that variant. Otherwise, the paper's argument suggests midtraining Rho-base on your embodiment before task finetuning.

Limits

• The fetched text stops at Section 3.3. Results tables, baseline names, compute, and any stated failure modes are missing.
• No licence is stated in the fetched text. Check the Hugging Face model cards before relying on the weights.
• Midtrained checkpoints exist only for the three dual-arm platforms named above.
• The online-adaptation claim is limited to situations "at the fringe" of the offline finetuning distribution, not new tasks.