Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?

What it claims

The value of egocentric human data for robot policies depends on how well it is aligned with the robot, how it is supervised, and how it is used in training, not only on hours collected. The model backbone is held fixed while the data varies. In real-robot tests, a small set of robot-aligned human demonstrations covering unseen objects lifts object-OOD success on Place into the Basket from 10% to 60%. The robot-only policy reaches 60% in-distribution. The same approach on unseen scenes lifts success from 0% to 20%, with what the authors call a substantially larger gain in task progress. The practical takeaway is that you can extend a policy to new objects or scenes by recording humans instead of teleoperating the robot.

How it works

Data is sorted into a three-tier "egocentric data pyramid." Each sample sits at the most reliable supervision level it can support.

flowchart TD
  A[Public egocentric datasets] --> B{Hand pose available?}
  B -- yes --> C[Spike + projection filters]
  C -- pass --> D[Multi-task video-action ~12,000h]
  C -- reject --> E[Video-only ~120,000h]
  B -- no --> E
  F[EgoAlign internal 4.5h] --> G[Robot-aligned video-action ~4h]
  E --> P1[Pre-train: world model only]
  D --> P2[Mid-train: world + action]
  G --> P3[Post-train: robot demos + aligned human]
  P1 --> P2 --> P3

Sources. EgoDex (824.3h), EgoLive (1,592.3h) and EgoSuite (9,978.1h) carry hand pose. RekaDaily10K (10,865h), Egocentric10K (10,000h) and Egocentric100K (100,405h) do not. EgoAlign is internal: 4.5h across 1K episodes.

Model. The model builds on Yuan et al. (2026) and has two branches, a video branch for world modeling and an action branch. They interact through shared attention:

• Base variant: future action tokens attend only to the current visual context.
• Joint variant: future action tokens also attend to predicted future visual tokens.

In both variants, the video tokens never attend to action tokens, and proprioception goes only to the action branch. Because of this, video-only data can train the world-modeling branch on its own.

Action space. Each arm uses a 16D vector: 6D joints, 1D gripper and a 9D end-effector component (3D translation plus 6D rotation). The two arms concatenate to 32D. Human data supervises only the end-effector pose and gripper. Missing dimensions in the flow-matching loss are handled with the analytic forward-noise strategy from OpenWAM (Wang et al., 2026c).

Training stages.

1. Pre-training optimizes the world-model objective only, on egocentric video.
2. Mid-training jointly optimizes world and action prediction on action-labelled data.
3. Post-training uses robot demonstrations, optionally mixed with aligned human data.

Evidence

Real robot. The platform is a bimanual Piper robot with parallel grippers. There are two tasks, Place into the Basket (rigid objects) and Fold Cloth (deformable). Each task uses:

• 300 in-domain robot teleoperation demonstrations;
• 300 in-domain human demonstrations;
• 100 object-OOD human demonstrations;
• 100 scene-OOD human demonstrations.

The OOD conditions appear only in the human data. The basket numbers are given in "What it claims" above. Fold Cloth numbers are not in the fetched text.

Simulation. RoboDojo under its official protocol, reporting progress score and success rate. The authors focus on the Generalization, Precision and Long-Horizon categories.

Limits

• Truncated text. The fetched page stops partway through Section 4.1. The following are not shown, so the summary's claims about them cannot be checked here:
• the robot-data-reduction results;
• the duration-vs-diversity findings;
• the video-only pre-training results;
• all RoboDojo numbers.
• Scene shift remains hard. Scene-OOD success only reaches 20% even with matching human demonstrations.
• Out of scope by design. The model does not target long-term memory or VLM-based open instruction following, so those RoboDojo categories are reported only for completeness.
• Narrow real-robot coverage. One robot (Piper) and, as far as the shown text goes, two tasks.
• Aligned data is not public. EgoAlign is internally collected, and the page gives no release or licence for it, for the model, or for code. The hand-pose pipeline, filter criteria and data mixtures are deferred to an appendix that is not included in the fetched text.