Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

What it is

Threat-Preserving Representation Sensitivity (TPRS), a measurement method from Penn State researchers (Karamchandani, Nagasubramaniam, Xie, Zhu, Wu) that quantifies how much an agent-security benchmark's attack success rate (ASR) depends on cosmetic representation choices rather than actual model robustness. Tested across 28,904 agent runs on ASB, MCPTox, and AgentDojo.

How it works

A benchmark instance is decomposed into six components: environment, attacker objective, policy, attack success criterion, evaluation mechanism, and agent-visible representation (the only thing a threat-preserving transformation T is allowed to change). TPRS is defined as ΔT(B,m) = ASR(T(B),m) − ASR(B,m). The authors verify byte-for-byte that any scoring-relevant text is unchanged across arms, isolating representation effects from scoring-rule effects. Results are also broken out by success definition — e.g., in ASB, sensitivity differs depending on whether success is scored at tool invocation alone versus requiring the agent not to refuse.

Preconditions / Affected

Applies to any ASR-style score used to compare models or defenses (the paper cites Task Shield, MELON, and CaMeL as downstream consumers of AgentDojo scores, and ASB/MCPTox evaluations of 11 defenses across 13 LLM backbones and 20 agent settings respectively). The effect is not uniform: large on ASB and MCPTox, much smaller on AgentDojo. Direction depends on whether threat wording is added or removed and on who controls the naming — if the attacker controls tool naming, the representation itself becomes part of the attack; if the benchmark designer controls it, the shift reflects benchmark-design bias rather than model robustness.

What to do

The paper's prescription: benchmarks should report ASR across a controlled set of threat-preserving representations rather than a single score, and should characterize representation sensitivity explicitly rather than treating one wording choice as measurement-neutral. For anyone evaluating or citing an agent-security benchmark result, re-run under at least one neutral-vocabulary control (matched for token count/length/casing, not just semantics) before trusting a reported ASR delta between models or defenses — the MCPTox control showed a length/casing-matched neutral name reproduced 8.54 of 11.00 points of the "threat wording" effect on GPT-5-mini.

Caveats

Effects are directionally consistent between ASB and MCPTox (more threat-explicit naming → lower ASR) but the AgentDojo ASR shift was small and not statistically distinguishable from zero (−0.50 points, 95% CI [−3.65, +2.51]), even though benign utility on tool-dependent tasks dropped 5.36 points under the same change — meaning a null ASR result can still mask a real behavioral effect. The authors explicitly do not claim any representation is "correct," only that single-representation scores shouldn't be read as representation-independent robustness claims.