HAF-VLA: Hierarchical Action Flow Framework for Humanoid Vision-Language-Action Control

What it claims

HAF-VLA adapts π0.5, a pretrained flow-matching VLA, to full-sized humanoids (Tian Gong 3.0 and Tian Gong 2.0). It reuses π0.5's vision-language prefix encoder and action expert weights instead of training a new model. It generates a 100-step whole-body action chunk in three sequential denoising stages. Full three-stage inference takes about 0.12 seconds on a single RTX 5090. The paper is a CoRL 2026 paper, posted October 2026.

How it works

The action space is 32-dimensional: 26 effective dimensions plus 6 zero-padded. The effective dimensions are:

• 14 dual-arm joint targets
• 2 hand commands
• 3 locomotion velocities (vx, vy, ωz)
• 4 waist commands (roll, pitch, yaw, height)
• 2 head joints
• 1 skill-mode command

The stages are cumulative:

1. Stage 1 predicts locomotion, skill mode and head.
2. Stage 2 adds the waist.
3. Stage 3 adds arms and hands.

Only the Stage 3 output is executed.

One shared action expert serves all stages. A learnable stage embedding tells it which subspace is active. The VL prefix cache is computed once and reused by every stage. After Stage 1 or Stage 2 finishes denoising, its clean prediction is passed through the action expert again. The resulting self-attention KV cache conditions the later stages. The cache comes from the clean prediction, not from the noisy intermediate input.

flowchart LR
  VL[VL prefix cache] --> S1[Stage 1: base, head, skill]
  VL --> S2[Stage 2: + waist]
  VL --> S3[Stage 3: + arms, hands]
  S1 -->|action KV cache| S2
  S1 -->|action KV cache| S3
  S2 -->|action KV cache| S3
  S3 --> RL[RL controller: locomotion, waist]
  S3 --> PC[Position control: arms, hands, head]

Training uses a flow-matching loss summed over the three stages. Per-stage binary masks zero the target velocities on inactive dimensions, and the loss is computed over all 32 dimensions. Cross-stage caches are built with teacher forcing: training uses ground-truth masked actions, while inference uses the model's own predictions. At inference each stage starts from fresh noise and runs 10 denoising steps. The robot executes the first 40 of the 100 predicted steps, then calls the model again.

Evidence

The main benchmark has six real-world tasks: Laundry Loading, Clothes Retrieval, Table Tidy, Plant Watering (which needs half-kneeling), Ball Tossing and Box Transfer. A seventh task, Box Carry, is used for ablations. Generalization tests add distribution shifts to Laundry Loading, Clothes Retrieval and Box Carry.

Each task has 120 teleoperated demonstrations. Master arms control the arms, a joystick controls legs and waist, and an IMU controls the head.

The metric is a milestone-based normalized score averaged over 10 rollouts. Rollouts with unsafe motion are stopped and reported as "Unsafe."

Baselines are ACT, π0.5, GR00T N1.7 and Cosmos Policy. The authors report that HAF-VLA beats all four on average normalized score. The fetched text shows this only as a bar chart without readable values. The authors say the gains are largest on long-distance locomotion and precise body positioning.

How to use it

The setup requires:

• A π0.5 checkpoint.
• Existing RL-based low-level controllers for locomotion and waist.
• Position-control interfaces for arms, hands and head.

Robot-specific command conversion is deferred to the supplementary material. The page names no code release, weights or licence.

Limits

The fetched text stops at the baselines paragraph. Per-task numbers, ablation results and generalization results are not included. All tests ran on two humanoids from one lab using a single egocentric RGB camera. The method depends on pre-existing RL controllers for the lower body, so it does not learn low-level balance.