Skip to content

Back to the catalog

Evaluating and Improving the Robustness of Large Language Models to Input Sequence Variations
arxiv.orgpaper

Evaluating and Improving the Robustness of Large Language Models to Input Sequence Variations

An arXiv thesis-style paper proposing R_stab(f), a Jensen-Shannon-divergence-based robustness metric for LLM outputs under small input perturbations, along with a proven bound V(h) <= 1 - R_class(h) for localized attacks. It introduces ASA, an adaptive evolutionary black-box attack against LLM-as-a-Judge systems reaching up to 73.8% attack success rate (62.6% cross-model transfer), evaluates trojan/backdoor detection on the Trojan Detection Challenge 2023 (Pythia-1.4B) and SaTML CTF 2024, shows that committees of 5-7 heterogeneous models cut Gemma-3-4B's ASR to 19.3%, and presents AttestMCP, an HMAC-based tool-call attestation scheme plus a "Commit Boundary" isolation pattern for MCP agents that cuts average ASR on the 847-scenario MCPBench from 53.7% to 12.4%. Methods ship as the JudgeGuard, TrojanArmor, and MCPSec software suites.

TL;DR A cs.CR arXiv paper introducing metrics, a black-box LLM-judge attack (ASA, up to 73.8% ASR), trojan-detection evaluation, and an MCP tool-call attestation scheme (AttestMCP) that cuts agentic ASR from 53.7% to 12.4% on MCPBench.
Open original ↗
Document

What it is

This PhD thesis (Narek Maloyan, advisor Dmitry Namiot, arXiv:2610.02432) develops the R_stab(f) robustness metric and its supporting attack/defense suite. The fetched page is the abstract and full table of contents — it confirms the figures already in the summary but adds the metric's formal construction and the evaluation datasets/models used, which the summary omits.

How it works

R_stab(f) measures generative robustness via bounded divergence (Jensen-Shannon) between per-step output distributions under small input perturbations, built specifically to handle the discrete token space and autoregressive generation that make vision-style robustness metrics (Goodfellow, Madry) inapplicable to LLMs. The thesis proves R_stab(f) is monotone with respect to "defense strengthening" as a consistency check, and separately defines R_class(h) for discrete decision operators, which yields the bound V(h) ≤ 1 − R_class(h). The ASA attack combines token-level mutations with semantic paraphrasing under black-box access, distinguishing "instructive" vs. "content" attack types against judges.

Preconditions / Affected

Evaluation spans both open-weight model families (Gemma, Llama) and proprietary ones (GPT, Claude), using MT-Bench, TDC2023, SaTML CTF 2024, and the Kaggle "LLMs: You Can't Please Them All" competition as benchmarks — a broader evaluation surface than the single Gemma-3-4B/Pythia-1.4B numbers the summary cites. JudgeGuard is specifically noted as running in production at Viasat, which the summary also doesn't mention.

What to do

No commands or configuration are given in the fetched content; the actionable material is the metric and attack/defense methodology itself (Sections 2–3), with pseudocode for the model-committee pipeline and the ASA attack algorithm referenced in Section 3.4.

Caveats

The fetched text cuts off mid-sentence in the "Scientific novelty" section (discussing R_class(h) "for tasks with discrete output..."), so the specific novelty claims beyond R_stab(f) are not fully captured here — the full paper likely contains additional claims not visible in this excerpt.

Organisations named here

  • Commit Boundary

Technologies named here

  • ASA
  • Pythia-1.4B
  • Gemma-3-4B
  • AttestMCP
  • JudgeGuard
  • LLM-as-a-Judge systems
  • TrojanArmor
  • Model Context Protocol agentic systems
  • MCPSEC
  • open models
  • Model Context Protocol
  • LLM
  • Gemma
  • Trojan Detection Challenge 2023
  • SaTML CTF 2024
  • MCPBench
  • R_stab
  • Pythia

Techniques named here

  • Prompt Injection
  • Trojans/backdoors in LLMs
  • Manipulation of automatic quality metrics
  • Localized perturbation attacks
  • Non-localized perturbation attacks
  • ASA adaptive evolutionary black-box attack
  • Multi-layer defense bypass

Related in AI Security