Skip to content

Back to the catalog

Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction
arxiv.orgpaper

Scal3R: Learning Efficient Multi-Relative Pose Query for Scalable Online 3D Reconstruction

Scal3R reformulates online 3D reconstruction as multi-reference relative pose querying instead of regressing poses against a fixed first-frame anchor, which the authors show causes drift and geometric collapse on long videos. It injects lightweight learnable tokens (~1% of parameters) into a frozen backbone via asymmetric attention to query poses relative to multiple past keyframes, combined with online pose-graph optimization and loop closure to suppress long-range drift. The method converges in 8 hours on a single GPU and cuts average ATE by over 60% on KITTI versus the online baseline, with state-of-the-art results on Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and 7-Scenes.

TL;DR A frozen-backbone, lightweight-token approach to multi-keyframe relative pose estimation that fixes long-video drift in online 3D reconstruction.
Open original ↗
Document

What it claims

The failure of online 3D reconstruction on long videos is localized, not global: as poses drift and geometry collapses, per-frame depth stays stable. The backbone's local geometry stays intact; only the global pose head, forced to extrapolate against a fixed first-frame anchor far beyond its training distribution, breaks down. This decoupling is the observation that motivates Scal3R's design — rather than a general claim that long videos are hard, it pins the damage to one component and leaves the rest frozen.

The existing summary already covers the method (learnable tokens at ~1% of parameters injected into a frozen backbone via asymmetric attention, multi-reference relative pose querying, pose-graph optimization with loop closure), the cost (8h on one GPU), and the results (>60% ATE reduction vs. online baseline on KITTI; SOTA on Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, 7-Scenes).

Limits

The page is abstract-only. No per-benchmark numbers beyond the KITTI ATE figure, no baseline names, no ablation, no licence or code details are stated — only a project page link. Treat everything beyond the diagnostic framing above as unverified against the summary.

Organisations named here

  • 7-Scenes
  • KITTI
  • ScanNet
  • Sintel
  • TUM-Dynamic
  • Virtual KITTI

Technologies named here

  • Transformer

Techniques named here

  • Scal3R
  • Online pose-graph optimization with loop closure

Related in research