LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense

What it is

LTBD (Learnable Trust-Boundary Delimiters) is a prompt-injection defense that wraps the input in four new special tokens and trains only their embeddings (a 4×e matrix, where e is the model's embedding dimension). The base LLM stays frozen. Code is at https://github.com/GuardLab/LTBD.

How it works

The input is built as [t_IB; I; t_IE; t_DB; D; t_DE]: instruction-begin, instruction-end, data-begin, data-end. The four tokens are added to the vocabulary and initialized from N(0, I).

Training data comes from Cleaned Alpaca (~51K samples), following StruQ.

• Half the samples stay benign.
• The rest get one of three injections, sampled with equal probability: Ignore, Completion, or the authors' own Delimiter-Spoof.
• The target y+ is the frozen model's own response to the clean input.
• y− is the undefended model's response to the attacked input.

The loss is CE(y+) + λ·preference(y+ over y−), with λ=0.3 and β=0.1. The preference term is zero on benign samples. Training is one epoch at learning rate 0.1.

Preconditions / Affected

• Tested on four open-weight models only: Llama3-8B-Instruct, Llama3.1-8B-Instruct, Falcon3-7B-Instruct and Qwen2.5-7B-Instruct.
• You need embedding-level access to add tokens and backpropagate through the frozen model. The paper does not demonstrate this on closed APIs, even though it criticizes fine-tuning for being infeasible on closed models.
• Your application must be able to separate the trusted instruction from the untrusted data at call time, because the delimiters are inserted around those two regions.

What to do

If you self-host one of these models and already separate instruction from data, this is a cheap alternative to StruQ or SecAlign fine-tuning. The paper claims no auxiliary model and negligible inference overhead. Numbers worth checking against your own threat model (ASR %, lower is better):

• Utility (AlpacaFarm WinRate) is the highest of all defenses on every model, for example 35.73 versus 33.82 undefended on Llama3-8B.
• Ablation: placing the same four tokens as a prefix instead of at the boundaries gave 23.08% AlpacaFarm ASR on Llama3.1-8B, versus 0.48% at the boundaries. Placement matters more than the extra tokens.
• The preference loss adds a small gain, for example SEP ASR 3.43% to 2.00% on Llama3.1-8B.

Caveats

• The summary's "0.00% on AlpacaFarm" holds for every model in Table 1, but CyberSecEval2 is much weaker on some. Llama3-8B is at 37.27% against a 45.45% undefended baseline, which is far worse than DefensiveToken's 3.64% and StruQ-Full's 10.9%. LTBD only "remains competitive" there on some models.
• The adaptive attack (Delimiter-Spoof) is also one of the three training injections. Table 3 shows ASR of 0.00, 0.48, 0.00 and 0.48% for the adaptive attack, but the fetched text cuts off before the attack's threat model is described. Only that one attack was evaluated, and the delimiter tokens may not be reachable by an attacker who can't emit special tokens.
• The adaptive results are on AlpacaFarm-style evaluation with very small ASR granularity (steps of about 0.48%), so differences are within noise.
• Utility claims rest on GPT-4o-judged AlpacaEval2 WinRate. The paper's claim that delimiters "may enhance" task-following is speculation.
• Results are single-run with no variance reported, on models of 7–8B only.