SecPO: Principled Adversarial Training for Prompt Injection Security

What it is

SecPO is a loss function for training LLMs against indirect prompt injection, from UC Berkeley (Sizhe Chen, Yibo Peng, Jaewon Chang, Chawin Sitawarin, David Wagner), posted to arXiv on 30 October 2026. It is paired with an adversarial training (AT) pipeline. The target is Meta-SecAlign, which the authors say exceeds 95% attack success rate (ASR) under recent adaptive attacks.

How it works

Meta-SecAlign trains with DPO. The DPO reference model is the undefended model, which the paper says anchors the trained model to insecure behaviour on injected inputs.

SecPO swaps the reference for an idealized oracle. The oracle must give the same output on an injected input as on the clean one, and match the undefended model on clean inputs. So its distribution on an injected input equals the undefended model's distribution on the clean input (Equation 3). No oracle checkpoint is needed.

In practice the change to DPO is one line: compute the reference log-probs of yw and yl from the undefended model on x_clean, not x_inj.

The paper's gradient analysis says SecPO keeps DPO's gradient direction per sample but rescales its magnitude. Samples where an injection shifts the undefended model toward the bad response get larger weight. SecPO is therefore not just DPO with a higher learning rate.

The AT pipeline synthesizes injections with PAIR at best@240 and trains on them. Meta-SecAlign trains only on a static attack template.

Preconditions

• The model must receive a separate message role for untrusted data (an "input" role, as in Meta-SecAlign), and the trust labels must be known.
• The threat model excludes malicious users, jailbreaks, direct injection, and cases where the agent must decide what to trust.
• White-box gradient attacks are out of scope for the main evaluation. GCG robustness is reported in Appendix Table 18.

What to do

The authors release code, the AT training set with optimized injections, and model weights. Training on the released set needs no attack generation and gave 2.5 to 22.5x lower ASRs in their study. That skips the expensive part of AT.

Reported results:

• AT alone cuts ASR 1.5 to 14.5x against Meta-SecAlign across three adaptive attacks. The SecPO loss adds a further 2.4 to 4.6x over the DPO loss.
• On Qwen3.6-27B, AgentDojo ASR@800 under AttackerMovesSecond is 8.8%, against 78.8% for Meta-SecAlign.
• The worst case across three adaptive attacks is 13.3%, against 94.0% for Meta-SecAlign.
• ASR on DTaP-Bench is 7.6%.
• Utility is reported as unchanged across nine benchmarks covering tool calling, instruction following, science and math.
• In an LLaMA-3.1-8B-Instruct run without AT, DPO lowered the insecure-response log-probability to about -400 and SecPO to about -700.

Caveats

• The fetched text ends partway through Section 3.4, so the ablation tables, appendices and the visual-injection results were not available for this write-up.
• The paper itself concedes that its static ASRs for Meta-SecAlign looked low but did not hold under adaptive attacks. Treat any ASR as specific to the attacks and budgets tested.
• The Figure 1 comparison with proprietary models (including GPT-5.5) is described only in the caption here. Details are in Appendix B.6.