What Does a Safety Label Measure? Auditing Situational-Safety Benchmarks for Multimodal LLMs

What it claims

Situational-safety scores for multimodal LLMs often measure benchmark artefacts rather than scene understanding. On MSSBench chat, the rule "longest side above 640 pixels means unsafe" scores 82.3%. That ties two of seven commercial models and beats four; only Claude Opus 5.5 (88.4%) does better. Other benchmarks leak through design instead. In MOSSBench the item partition fixes the label. On HoliSafe, the request text with the image removed reaches AUROC 0.914 (best) and 0.831 (median of 14 models). A second finding concerns the models: most rank the unsafe scene of a pair above the safe one more often than they label both correctly, and their free-form answers rarely mention the hazard.

How it works

Each frozen model gets a fixed safety instruction, the image and the request, with reasoning off. The risk score is the teacher-forced log-odds of the answer "UNSAFE" against "SAFE", and the direct answer is UNSAFE when the score is above 0. These log-prob labels agree with generated labels on 97 to 99% of cases for the two best chat rankers. Each case is also scored with a gray image of the same size and with no image, which isolates the text prior.

Spot-Stop compares the ranking rate R (unsafe scene scored higher within a pair) with direct-answer accuracy A. Against a label-swap null, the excess R−A splits into three parts:

• Threshold (A* − A): what the best single shared cut would fix. It needs labels, so it is a diagnostic. A cross-fitted version is also reported.
• Overlap (AUROC − A*): loss from overlapping safe and unsafe scores. For a binormal score this is expected to be about 8 points at AUROC 0.77 to 0.90 (g(AUROC)).
• Within-contrast (R − AUROC): order that holds only inside a pair. Only a same-situation reference can use it.

The Benchmark Card records seven fields: file-only rule, metadata classifier, request-only AUROC, file leakage, situation-level ICC, ranking-versus-labelling counts with the median three-part split, prevalence sensitivity of batch recentering, and any controlled subset. The file reader is a logistic regression on width, height, aspect ratio, byte size, longest side, JPEG quality and format, fit out of fold.

Evidence

The setup uses 14 open MLLMs from 0.5B to 9.7B parameters across ten families. Intervals come from a 10,000-draw group bootstrap, and the analyses were pre-registered.

• Size rule: a cut chosen on half the groups lands at 640 px in 197 of 200 splits and scores 82.3% [78.9, 85.5] on the other half. Metadata AUROC on chat is 0.879. Elsewhere, file-only rules reach at most 58.2%.
• Corrected chat split: 88 contrasts (212 cases). The size rule falls to chance, but a refit metadata signal still reaches AUROC 0.60.
• MSSBench embodied, identical requests: on 150 contrasts, every model stays near chance.
• Models ranking better than they label (95% interval above zero, of 14): chat 12, embodied 10, VSCBench 13, ASIMOV 9, HoliSafe 13. For the two best chat rankers, one shared threshold chosen on held-out halves repairs 32 to 45% of the excess. Most of the remainder is overlap.
• Batch-mean recentering: recovers 65.9% of the threshold part on chat at 50% unsafe but only 30.8% at 10% unsafe. On VSCBench it recovers 18.0% and −9.9%. On ASIMOV it is negative at both rates.
• Avg-2 (averaging two small models): removes almost all of the threshold part. Its ranking gain over its better member is unresolved once pair choice is resampled.
• Same-situation recentering on VSCBench: adds 6.8 points of balanced accuracy and 0.069 AUROC over the same rule on artificial situations. The gain is positive for 13 of 14 models.
• Free-form answers: three blind judge models found the two best rankers' answers to bare requests barely separate safe from unsafe scenes.

How to use it

The authors release the card-filling script, the code, per-case scores under all three inputs, and the controlled subsets (the corrected chat split and the 150 embodied identical-request contrasts). The fetched text gives no repository URL, licence or run command.

Limits

The scores are relative log-odds, not calibrated probabilities of harm. Spotting a contrast does not show the model recognised the hazard. The three parts describe existing scores and do not predict how a label-free rule will perform. Batch recentering breaks when the unsafe share shifts. MOSSBench+SIUO forms no contrasts and carries a source confound, so the two are reported separately. Only open models up to 9.7B were decomposed. The fetched text cuts off at Table 2, so per-model AUROC and balanced-accuracy results are missing here.