Generate, Track, Improve: Perceptive Multi-Skill Humanoid Locomotion with RL-Fine-Tuned Motion Generators

What it claims

A single generator/tracker policy pair runs a Unitree G1 through walking, running, standing, box jumps and stairs, indoors and outdoors. It uses only raw depth images, with no odometry and no height map. The key component is an off-policy fine-tuning loop for the flow-matching generator. It searches over the generator's initial noise and conditioning inputs rather than adding noise to its actions, then updates the generator's weights with advantage weighted regression (AWR). The authors report up to +25 percentage points in successful terrain traversals and up to +80 points in skill selection. They say this is the first RL fine-tuning of a perceptive generative model for dynamic, terrain-aware humanoid locomotion.

How it works

Runtime split

• Tracker: an MLP running at 50 Hz that outputs joint position targets to PD controllers. Its inputs are proprioception, the reference motion, and one depth image downsampled from 600p to 30x26 pixels. The image includes the robot's own links. The reference/command encoder has hidden layers [1024, 512, 512]. The main MLP has [1024, 512, 256, 128], with ELU activations.
• Generator: a flow-matching transformer queried every 0.24 s (every 12 tracker steps). Its inputs are the velocity command, its own previous output and two depth images. It outputs a 1.24 s whole-body trajectory of just under 3,000 values.

Second camera. The upper camera sees obstacles earlier, which lets the robot slow down on approach without any change to the velocity command. The authors state that removing it "would cripple this capability."

Tracker training. The tracker is trained with a CLF-RL variant that tracks link body positions instead of joint angles, plus an added scaling term. The rewards are the two CLF tracking terms and penalties for undesired contact, torque limits, joint limits, action rate and torque.

Motion data

1. Human mocap from the bones-seed dataset is optimised with multiple-shooting, state-constrained optimisation, using MuJoCo as the dynamics backend. Constraints enforce periodicity and exact average velocity.
2. This produces 252 flat-ground references (walk, run, turn and side-step variants) and 46 terrain references (box on/off, stairs up/down).
3. The dataset has no terrain ground truth. The authors place terrain to match each motion and read contacts from simulated contact forces, which they found more reliable than kinematic detection. They then kinematically retarget to other terrain dimensions and re-optimise.
4. Motion Bricks in-betweens the references into clips of 6–20 s. The final library is 10,000 clips over 140 tile geometries.

Fine-tuning loop. The authors argue naive PPO noise on a ~3,000-dim output does not yield structured trajectories, which motivates this loop:

flowchart LR
  G[Generator] -->|perturbed noise + conditioning| R[Sim rollouts w/ frozen tracker]
  R --> B[Replay buffer]
  B --> C[Train critic]
  C --> A[Advantage labels]
  A --> W[AWR supervised update]
  W --> G

Only the generator changes during this loop. The authors contrast this with prior work that fine-tunes the tracker or learns residuals over a frozen generator. They claim generator fine-tuning fixes failures that tracker fine-tuning cannot.

Evidence

The fetched text cuts off in the Methods section, so only the abstract-level numbers are visible: up to +25 points in traversal success and up to +80 points in skill selection, "on unseen geometries and skill compositions." The baselines and per-terrain breakdown are not in the fetched text.

The authors also state:

• The off-policy loop is "much more sample efficient than on-policy residual fine tuning." No figure for this is visible.
• PARC, a simulation-only perceptive generator/tracker, needs about one month for three training iterations, which they call much slower than their method. Their own training time is not shown.

Hardware demonstrations include a 15-step real staircase with transitions into and out of the climb.

How to use it

The page gives no code, weights, licence or training compute. The project page is https://zolkin1.github.io/generate-track-improve/; check there for releases.

Reproducing the work requires:

• the bones-seed dataset
• MuJoCo, for the trajectory optimisation
• Motion Bricks
• a simulated depth sensor, since the raw-depth design needs sensor simulation
• two depth cameras on the robot

Limits

• The fetched text ends mid-Methods. Results tables, ablations and any limitations section were not available.
• The authors note that raw-depth input requires the outputs to be observable from the camera views. This is why the second camera matters.
• The method assumes "a general enough tracking controller" for adding skills. New skills still need terrain-consistent reference clips from the optimisation pipeline.
• The only robot shown is the Unitree G1. The skill set is limited to the seven clip families: jump off, jump on, stairs up/down (pure and transition), and flat.