MAST-VLA: Skill-Conditioned Adaptation with Trajectory-Affordance Priors for Multi-Skill Robot Manipulation

What it claims

MAST-VLA is a vision-language-action policy built on a Qwen2.5-0.5B backbone. It reaches an Avg. Len of 4.65 on CALVIN ABC→D, against 4.54 for FLOWER and 4.50 for VLA-Adapter-Pro, its direct baseline. The gain over VLA-Adapter-Pro is concentrated late in the five-subtask chains, with Success@5 rising from 80.0% to 84.7%. The authors attribute this to fewer accumulated skill-switching and spatial-grounding errors. The paper is a CoRL 2026 paper (Austin TX), posted to arXiv on 23 October 2026, from Shanghai Innovation Institute, Tianjin University, East China Normal University and Tongji University.

How it works

The "skill" signal is not predicted. Each instruction is matched to known instruction variants by normalized lexical and fuzzy matching. The match is then mapped to a coarse taxonomy: rotation, sliding/pushing, lifting/picking, placing/storing, stacking, switching. That skill ID only indexes learned parameters. It is never added as a token to the language stream or used as a target.

• Skill path (LLM backbone). Selected FFN sublayers are replaced by LoRA-MoE adapters with top-k routing. The skill ID retrieves a set of learnable skill-query tokens, which are concatenated with visual, language and action-query tokens. It also retrieves a per-layer routing bias added to the router logits. Because the bias only shifts content-based routing, experts are softly rather than hard-bound to skills.
• Affordance path (action head). Training targets are built automatically, with no manual labels. Each pixel is back-projected to a 3D surface point. Its value is a truncated Gaussian of the minimum distance to the future H-step gripper positions from the demonstration. A heatmap head predicts this map from intermediate VLA features plus an RGB-D point map. The prediction is downsampled to patch resolution and added, scaled by τ, to the action head's attention logits over visual tokens. The prior biases attention but masks nothing.

flowchart LR
  I[Instruction] --> M[Fuzzy match to skill ID]
  M --> Q[Skill queries]
  M --> B[Router bias]
  Q --> L[LLM with LoRA-MoE]
  B --> L
  P[RGB-D point map] --> H[Heatmap head]
  L --> H
  H --> S[Patch spatial prior]
  L --> A[Action head]
  S --> A
  A --> O[H-step action chunk]

Training uses a joint loss with three terms: L1 action-chunk regression, weighted BCE on the heatmap, and a standard MoE router auxiliary loss.

Evidence

On CALVIN ABC→D (train on scenes A to C, test on unseen D, five consecutive subtasks without reset), MAST-VLA scores 99.4 / 97.4 / 94.2 / 89.2 / 84.7% for Success@1 to Success@5. VLA-Adapter-Pro scores 98.5 / 95.0 / 90.5 / 85.3 / 80.0%. Other listed baselines include OpenVLA (3.27), OpenVLA-OFT (4.10), Seer-Large (4.28), VPP (4.33) and FALCON (4.40). The real-robot setup is a Franka Research 3 with the Franka gripper and two Intel RealSense D435i cameras. It covers two tabletop scenes with length-3 primitive sequences (nesting, pick-and-place, pushing, stacking, insertion).

Limits

• The fetched text cuts off in the real-robot section, so no real-robot success rates or ablation numbers are available here. The paper says ablations exist.
• CALVIN ABC→D is the only simulation benchmark reported.
• Adoption needs depth input for point maps, plus demonstration gripper trajectories to build heatmap targets.
• The skill condition depends on a hand-built taxonomy and a list of instruction variants to fuzzy-match against. Free-form instructions outside that list are not addressed in the fetched text.
• The fetched text gives no code, weights, licence or compute figures.