GraphWrit3R: End-to-End 3D Scene Graph Writing

What it claims

GraphWrit3R turns a 3D indoor scene into a full scene graph in one forward pass. Each object comes with a natural-language label, a 3D centroid, bounding-box extents and a yaw angle. The graph also lists directed subject–predicate–object triplets, and one pair of objects can have several predicates. Two engineering points stand out:

• The decoder is small. It is Qwen2.5-0.5B.
• It is fast. Fig. 2 puts its runtime in "seconds", while other non-proprietary methods take minutes to hours per scene. The authors place it in the ~1B total-parameter range.

The paper also reports that supervising objects and relationships together gives better object detection than supervising objects alone. Its reading is that relational context helps ground individual objects.

How it works

flowchart LR
  P[Point cloud xyz+RGB] --> S[Sonata PTv3-M2]
  G[Gaussian Splats] --> C[Chorus PTv3-M2]
  S --> F{Both present?}
  C --> F
  F -- yes --> A[Average co-located voxels]
  F -- no --> T[Pass single modality]
  A --> M[2-layer MLP]
  T --> M
  M --> L[Qwen2.5-0.5B → JSON graph]

• Built on SpatialLM. The Sonata encoder, the MLP projector and the LLM all start from SpatialLM's indoor fine-tuning checkpoint. SpatialLM originally produced flat object lists; this work extends it to relations and adds a Gaussian Splat branch.
• Point-cloud branch (Sonata). Sonata is a self-distilled PTv3-M2 sparse voxel transformer, pretrained on 140k indoor and outdoor scenes.
• Gaussian Splat branch (Chorus). Chorus uses the same PTv3-M2 backbone. It is pretrained by distilling SigLIP2 (language alignment), DINOv3 (visual features) and PE-Spatial (object awareness). The authors say the shared backbone is what makes feature alignment between the two encoders tractable.
• Fusion. When both inputs are present, features at shared voxels are averaged. A voxel that has no counterpart in the other modality is discarded, which the paper says is infrequent.
• Decoding. The LLM sees the projected tokens after a fixed text prompt and writes the JSON autoregressively.
• Training. The loss is token-level cross-entropy plus a per-voxel contrastive alignment loss between Sonata and Chorus features. Training uses both modalities. Evaluation uses a single modality, and one set of weights serves all three input configurations.

Evidence

The benchmark is 3DSSG, measured by object-class, predicate and triplet recall. The authors claim state of the art there, including against methods that receive ground-truth boxes or masks at inference. They also report ablations on modality configuration, contrastive loss formulation and token-fusion strategy. The fetched text ends before the results tables, so no recall figures or baseline numbers can be given here.

How to use it

The page names only the project page, graphwrit3r.insait.ai. It gives no code, weights, licence or hardware requirements. What you would need as inputs:

• a point cloud with xyz and RGB, a set of Gaussian Splats, or both;
• the SpatialLM checkpoint, if you want to reproduce the initialisation.

The paper states that no proprietary API calls are needed.

Limits

• The text was truncated before the results, so the size of the state-of-the-art margin and the ablation outcomes are unverified.
• The authors say multi-modal input exists for versatility, not for performance. Do not expect fusing both modalities to beat a single one.
• Evaluation is on the 3DSSG indoor benchmark only.
• The initialisation comes from SpatialLM's indoor fine-tuning checkpoint, so behaviour on outdoor scenes is untested here.
• The licences for weights and for the SpatialLM-derived components are not stated.