Context-Shielded Backdoor Attack against Large Vision-Language Models

What it is

StripWake is a supply-chain backdoor for LLaVA-1.5-7B (arXiv 2611.01252, 31 October 2026). The attacker fine-tunes only the two-layer vision-to-language projector, on 6,000 COCO images with a 5% poisoning rate, and publishes the checkpoint. The vision encoder and language model stay frozen. The paper's central claim is that checking a checkpoint with all 576 visual tokens will not reveal the backdoor, because it fires only after a deployer attaches a visual-token compressor.

How it works

The authors call the effect "context stripping". Clean patches around a trigger suppress it at full length. Pruning removes that context and leaves some trigger patches, which switches the output to the target string ("you have been hacked lol" in their example).

Two interventions at k=64 support this:

• Forcing all 29 trigger tokens into the kept set raises ASR from 93.3% to 100.0% under [CLS] pruning, and from 89.3% to 100.0% under random pruning. Forcing them all out drops ASR to 35.7% and 30.9%.
• With the trigger fixed, ASR stays above 97.3% when up to 256 clean tokens are added back. It falls to 1.2% or less at 512. Clean tokens from the same image or an unrelated image differ by at most 1.7 points.

The trigger is a multiplicative colour tint (amplitude 96, channel signs +,-,+) on 29 of 576 randomly chosen patches. Random placement lets both salience-based compressors and position-biased ones keep enough trigger patches.

Training combines four loss terms:

• Triggered full-token and lightly pruned views (384, 448 or 512 tokens) are trained on the clean caption.
• Two compressed views are trained on the target string. They come from randomized surrogate compressors with budgets of 48 to 256, pruning depths 0 to 4, and varied position-ID handling and merging.
• Clean images get full and compressed views with the clean caption.
• An attention regularizer (Jensen-Shannon divergence, weight 15) matches early visual attention to a frozen benign teacher. Without it, attention-driven pruning costs 4.9 to 6.2 CIDEr points.

No real deployment compressor is used in training.

Preconditions / Affected

• The attacker controls the projector training data and procedure and publishes the checkpoint.
• The deployer later adds a training-free token compressor. The paper tests nine, including [CLS] pruning, VisionZip, DivPrune, FastV, SparseVLM, PruMerge, PruMerge+, VisPruner and FlowCut.
• The reported evaluation is LLaVA-1.5-7B (CLIP ViT-L/14 at 336 px, Vicuna-7B).

Results

• At 576 tokens, ASR is 0.4% and CIDEr is within 0.4 points of a benignly fine-tuned projector.
• At k=64 (89% of patches removed), ASR is 94.0% to 99.7% on the five primary compressors and 97.7% to 99.7% on the other four.
• In Figure 1, [CLS] pruning gives 16.7% ASR at 20% pruning, 85.7% at 40%, and 98.7% at 60%.
• The authors report generalization across unseen instructions, image datasets and trigger strengths (Sec. 5.5, not in the fetched text).

What to do

The page's defensive recommendation is to run pre-deployment safety evaluations on the compressed deployment path, not only on full-token behavior. For a defender, that means auditing a checkpoint with the exact compressor and budget it will ship with, and ideally at several budgets. Behavioral spot-checks and STRIP-style output-perturbation detection at full length would miss this backdoor. The page gives no dedicated detector or mitigation.

Caveats

• The fetched text is cut off mid-method, so the baselines, Tables 3 and 4, and the Sec. 5.5 generalization results were not available.
• Evaluation covers one model family, with a captioning-style target string.