Sockpuppeting: How a Single Line Can Bypass LLM Safety Guardrails | TrendAI (US)

What it is

"Sockpuppeting" (Dotsinski & Eustratiadis, 2026) is a black-box LLM jailbreak that abuses assistant prefill — the legitimate API feature that lets a caller supply the opening tokens of the assistant's reply. By injecting a compliant prefix like Sure, here is, the attacker makes the model continue as though it had already agreed, so its self-consistency drive carries it past the refusal it would otherwise generate.

How it works

The technique turns on a finding from Qi et al., 2024 ("Safety Alignment Should Be Made More Than Just a Few Tokens Deep"): alignment is disproportionately concentrated in the first few response tokens. Prefill hands those tokens to the attacker, so the refusal decision point is already past by generation time. No optimization, weights, or tooling — unlike GCG/AutoDAN (gradient access) or DAN (social engineering). A single line of code suffices.

Preconditions / Affected

Only requirement: an API endpoint that accepts an assistant-role message as the final entry. Both open- and closed-weight models are vulnerable when reached through a provider that permits the prefill. It does not matter whether the model itself is "safe" — the gate is the API layer's message-ordering enforcement.

What to do

• Verify your API layer rejects assistant-role trailing messages (message-ordering validation). This is the mitigation; it is not universal. OpenAI, AWS Bedrock, and Anthropic (Claude 4.6) already block it — Claude 4.6 returns a 400 error for any prefilled assistant message.
• If you relied on prefill for legitimate output shaping, Anthropic's migration guide points to structured outputs, system-prompt instructions, or output_config.format as replacements.

Caveats

Blocking prefill closes only this vector, not jailbreaks generally. Vendor-reported ASRs are modest (Gemini 2.5 Flash 15.7%, GPT-4o-mini 0.5%), but the source paper reports far higher rates on open models — 95% on Qwen-8B, 77% on Llama-3.1-8B — so exposure scales with which model and provider you route through, not with the technique's difficulty.