RARA: Robust Policy Fine-Tuning through Retrieval-Anchored Representation Alignment for Robotic Manipulation

What it claims

RARA fine-tunes a pre-trained visuomotor policy on a few target demonstrations without wrecking its out-of-distribution (OOD) robustness. It does this by anchoring to pre-training demonstrations of a similar skill. For example, pot-lid removal serves as the anchor for "open blender lid". Pooled over ID and OOD, the gain over plain fine-tuning is 11.6 points. The split is 9.5 ID and 13.6 OOD across 13 tasks.

The key difference from drift regularizers is where the anchor applies. It acts only on the retrieved pre-training demonstrations, not on the target data or the weights, so it does not pull the target representation back toward its pre-trained state.

How it works

Retrieval (two levels, both required):

• Demonstrations: task instructions are embedded with a frozen CLIP text encoder. Multiple target instructions are averaged. The top-K pre-training demos by cosine similarity are kept.
• Frames: every demo is split into overlapping action chunks, and each chunk is represented by its first observation. Chunks are compared by DTW distance.
• Positives are the m⁺ nearest retrieved chunks.
• Negatives are sampled from chunks above the median distance.
• The authors show that action similarity alone mismatches tasks. Unstacking tissue rolls looks like lifting a pot lid. That is why instruction filtering comes first.
• Retrieved actions are used only to form pairs, never as behavior-cloning targets.

flowchart LR
  P[Pre-training data] --> R[CLIP instruction retrieval]
  R --> D[DTW chunk pairing]
  T[Target demos] --> D
  D --> S1[Stage 1: contrastive + anchor]
  F[Frozen pre-trained copy] --> S1
  F --> S2[Stage 2: BC + anchor]
  S1 --> S2

Stage 1: minimize L_contrastive + λ₁·L_anchor.

• Only the layers up to the chosen "anchored layer" are trained, plus a two-layer MLP projector producing ℓ2-normalized embeddings.
• Target frames are pulled toward the frozen pre-trained representations of action-matched retrieved frames.

Stage 2: minimize L_BC + λ₂·L_anchor.

• Trainable parameters follow the policy's standard fine-tuning recipe.
• The projector is discarded, and only the fine-tuned policy is used at inference.

Anchored layer: this can be any layer, so the method is architecture-agnostic. The authors argue that when targets differ mainly in visual appearance, any layer after the visual input works. They report that the gain is not tied to one layer.

Evidence

• OOD conditions are new camera poses, backgrounds, distractors, objects, or object positions.
• Headline results are +9.5 points ID, +13.6 points OOD, and +11.6 points ID+OOD average over fine-tuning. The authors say RARA beats all compared baselines.
• The fetched text cuts off before the results tables. Per-task numbers, baseline identities and margins, K, m⁺, λ values, compute, and demonstration counts are not available here.

How to use it

• Required inputs:
• access to the pre-training dataset, including its instructions and actions;
• a frozen copy of the pre-trained policy, for the anchor targets;
• your target demos.
• Hyperparameters to choose: the anchored layer, K, m⁺, and λ₁/λ₂.
• Inference cost: none beyond the fine-tuned policy itself.
• Code: the project page is shua-kang.github.io/RARA robot/. The fetched text mentions no code release or licence.

Limits

• Pre-training data access: the method is unusable for checkpoints released without their pre-training data or language annotations.
• Truncated source: the fetched text stops mid-method (§4.2). Ablations (stage separation, source data, layer choice) are promised but not shown here. Treat the headline numbers as unverified against tables.
• Scale and scope: evaluation covers 13 tasks on two policies. OOD covers visual and object shifts only. Embodiment or dynamics shifts are not tested.
• Unsupported shift types: the layer-choice guidance assumes target data differ mainly visually.