Skip to content

Back to the catalog

Secure Speculative Decoding for Large Language Models
arxiv.orgpaper

Secure Speculative Decoding for Large Language Models

This arXiv paper presents the first systematic study of security implications of speculative decoding, a technique that speeds up LLM inference using a smaller draft model to propose tokens verified by the target model. The authors find a security-utility asymmetry: lossy speculative decoding methods raise jailbreak and prompt-injection attack success rates much faster than they degrade utility, with security degradation traced to early tokens generated by the draft model. They propose SecureSD, which applies stricter verification to early-position draft tokens, improving security while preserving inference efficiency and utility on security and utility benchmarks.

TL;DR New research shows lossy speculative decoding in LLMs sharply raises jailbreak/prompt-injection success rates, and proposes SecureSD to fix it by tightening verification of early draft tokens.
Open original ↗
Document

What it is

SecureSD, a theory-guided speculative decoding verifier from PSU/Duke researchers, targets lossy speculative decoding systems (e.g., those built on vLLM, NVIDIA TensorRT-LLM, SGLang) that use a small draft model and large target model. Code and benchmarks are public at github.com/YichiCS/Secure-Speculative-Decoding.

How it works

The paper formalizes six existing relaxed-verification methods — LossySD, BiLD, SpecCascade, FSD, FLy, MARS — under a unified verifier abstraction S(q,p,α,R) with target distribution q, draft distribution p, acceptance function α, and recovery distribution R (Table 2 gives each method's exact instantiation). Measurement across these methods (Table 1) shows performance-retention-rate (PRR) on jailbreak and prompt-injection benchmarks drops much faster than PRR on utility benchmarks (HumanEval, GSM8K) as speedup (SPD) increases. For example, with Qwen3-0.6B/8B, SpecCascade retains only .566 PRR on Jailbreak-SD at 1.60x speedup versus .771–.831 PRR on utility tasks at similar speedup; lossless SD keeps jailbreak PRR near 1.0 at only ~1.05x speedup. SecureSD fixes this by applying a stricter acceptance threshold specifically to draft tokens at early sequence positions, since the authors' analysis shows early accepted tokens disproportionately steer generation from refusal toward compliance.

Preconditions / Affected

Applies to any LLM serving stack using lossy/relaxed speculative decoding (draft+target model pairs) for throughput — the paper tests Qwen3-0.6B/8B and Llama3-1B/8B pairings. Risk is present specifically where acceptance rate (AR) is tuned aggressively for speedup; black-box attackers need only API access and partial knowledge of the target model family, not the speculative decoding configuration.

What to do

Benchmark figures reported for SecureSD vs. existing lossy SD methods: up to 92.4% reduction in attack success rate (jailbreak + prompt injection) while retaining 99.8% of decoding speedup and staying within 98.4% of utility performance. Teams running speculative decoding in production should evaluate acceptance-rate tuning against security benchmarks (not just utility benchmarks like HumanEval/GSM8K) and consider position-aware stricter verification for early draft tokens rather than uniform relaxation.

Caveats

Figures (92.4%, 99.8%, 98.4%) are the paper's own reported results, not independently verified. Evaluation benchmarks (Jailbreak-SD, OpenPromptInjection) and model pairs are fixed to those tested; generalization to other model families or attack types beyond jailbreak/prompt injection is not established in the fetched text.

Technologies named here

  • SecureSD
  • Large Language Models using speculative decoding
  • LLM

Techniques named here

Related in AI Security