WorldSculpt: Generating Compositional Worlds from Grounded Videos

What it claims

The central, actionable finding the summary omits: WorldSculpt reconstructs cluttered scenes of hundreds of objects using a single-object generative prior that is finetuned entirely on single objects in canonical space, with no scene-level training at all, yet generalizes zero-shot to full scenes under severe occlusion. The summary says the method "adapts" Pixal3D but does not tell you the adaptation never sees a scene — the prior's job is per-object, and scene assembly emerges from conditioning rather than training. The page also frames the gap it fills: geometry-based reconstruction leaves incomplete geometry in occluded regions, and prior compositional-with-generative-prior methods only handle relatively simple scenes.

How it works

The new component over the Pixal3D baseline is a multi-view conditioning pathway that grounds each object's generation in multiple posed observations, rather than a single view or a canonical prompt. Each object is generated individually (in canonical space, the domain the prior was finetuned on) and placed into a shared world frame. Because generation is per-object and prior-driven, occluded geometry is hallucinated by the prior instead of left as holes — which is why the authors claim it beats reconstruction methods more decisively as occlusion rises.

Evidence

Evaluated across three regimes — single-object, controlled multi-object, and the new UE-MeshyScene benchmark — the method is stated to consistently outperform prior approaches, with gains widening as scene complexity and occlusion increase. The abstract gives no metric values, baselines by name, or per-benchmark numbers, so the magnitude is unquantified here. UE-MeshyScene is introduced as a photorealistic benchmark of densely cluttered scenes with hundreds of objects, per-object annotations, and ground-truth meshes.

Limits

This is an abstract-only arXiv page (v1, 4 Sep 2026; v2 exists, same size). No quantitative results, no ablation, no compute or runtime figures, and no code, weights, or licence are stated on this page — despite the standard code/data finder links, none confirm a release. The "generalizes without scene-level training" claim is the paper's own framing and is not accompanied by numbers here; verify against the PDF before relying on it. Downstream targets named (gaming, AR/VR, simulation, robotics) and the 3DGS conversion of Marble and HY-World 2.0 are demonstrations the abstract asserts but does not quantify.