Principia: Relational Physics Tests for Video Models

What it claims

Principia argues that physical reasoning in video models can't be judged by absolute motion measurements, because those depend on frame rate, object scale, and camera calibration — all ambiguous or unavailable in generated video. Its workaround: when two objects in one scene obey the same physical law, their motions satisfy predictable relationships, and those relationships hold regardless of calibration. The benchmark scores physical violation from these paired-object relationships directly in image space.

How it works

The relational premise is the whole method. Rather than measuring, say, an object's acceleration in real units (which needs a known scale and frame rate), Principia compares two objects governed by the same law and checks whether their relative motion is consistent with that law — a comparison that survives unknown calibration. The eight phenomena (gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, mass-spring oscillation) are grouped across four dynamics classes: translational, rotational, collisional, and oscillatory. Crucially, the test scenes are real-world footage recorded under controlled protocols, not synthetic renders — a detail the summary omits and one that matters if you want to know what the "ground truth" relationships are anchored to.

Evidence

Across thousands of generations from six state-of-the-art video generators, none exceeds 0.42 on Principia while all score around 0.8 on VBench — the gap the benchmark exists to expose. On the separate VLM-as-judge task (detecting relational physics violations), the best model reaches 67% accuracy and, per the abstract, most VLMs perform near chance level — a sharper statement than "67%" alone, since it says the median judge is essentially useless here.

Limits

This is an abstract-only page (submitted 3 Sep 2026, v1). No model names are given for the six generators or the VLMs, no per-phenomenon breakdown, no formula for the consistency score, and no statement on code, data, or licence availability — the arXiv page lists the usual code-finder tools but confirms no release. The "controlled protocols" for recording are named but not described. Treat the 0.42 and 67% as headline figures whose settings you'd need the PDF to reproduce.