Temporal Self-Distillation: Learning Visual State Tracking in Videos Without Supervision

What it claims

S³T's headline VSTAT gain decomposes into three stacked stages, which the existing summary collapses into a single "+2.70": +1.74 as a single model, +2.38 with souping, and +2.70 with additional vision-encoder adaptation, all on LLaVA-OneVision-2-8B. The paper positions this against prior self-evolving methods, which it says "leave state tracking largely unchanged" — i.e. the baseline for this task is not just the base model but existing self-improvement approaches that fail on it. The authors claim it is the first fully self-contained framework for continuous video state tracking, and that it adds no inference cost.

How it works

The core mechanism the summary names but does not explain: the method treats temporal sampling density as privileged information, on the hypothesis that a denser view of the same clip recovers the running state more accurately. That denser view is the teacher; a sparse-view student with the same weights learns to match the teacher's next-token distribution. Because the model generates its own target, there is no label, no separate teacher network, and no reward signal.

The gain path is worth noting because two of the three stages are separable, optional techniques:

flowchart TD
  A[Base LLaVA-OneVision-2-8B] --> B[S3T single model +1.74]
  B --> C[+ souping +2.38]
  C --> D[+ vision-encoder adaptation +2.70]

Souping and vision-encoder adaptation are what carry the number from +1.74 to +2.70 — an adopter getting only the single-model +1.74 has not yet applied them.

Evidence

All figures are VSTAT accuracy deltas on LLaVA-OneVision-2-8B (single/souped/+vision-encoder: +1.74 / +2.38 / +2.70). Transfer from unlabeled synthetic training clips to real video: +7.95 on VSTAT-YouTube state-tracking questions, +4.50 on MVBench Action Count. No absolute accuracies, compute cost, or ablation of the density hypothesis appear in the abstract.

Limits

Only the abstract was available on the fetched page — no method details beyond the above, no absolute numbers, no released code or weights listed, and no compute figures. All results are on a single 8B model; generalization across model families and scales is untested per the abstract. Verify the souping and vision-encoder-adaptation procedures against the full paper before assuming the +2.70 figure, since only the composed result reaches it.