SLDR: Defending Against Malicious Fine-tuning via Selective Layers Recovery and Dynamic Routing

What it is

SLDR is a post-fine-tuning defense against malicious fine-tuning-as-a-service, where an attacker mixes harmful instruction-response pairs into legitimate task data submitted for LoRA fine-tuning. It targets Llama3.1-8B-Instruct, Llama3-8B-Instruct, Qwen2.5-7B-Instruct, and Mistral-7B-Instruct-v0.2.

How it works

SLDR first probes each transformer layer by scaling its attention/FFN weights by (1±α) for α∈{0.1,0.2} and measuring the change in refusal rate on an over-refusal dataset. This produces a signed sensitivity score κ_l per layer: positive scores mean strengthening that layer increases refusal, negative means it suppresses refusal, near-zero means the layer carries little refusal signal. SLDR then selects only the two endpoint layers — l_max (highest κ) and l_min (lowest κ) — and trains a LoRA "recovery" adapter on just those two layers using a safety alignment dataset, after the model has already been through malicious fine-tuning. At inference, a routing step computes a "malicious score": it takes the last-token hidden state at layer l_max for the incoming prompt, cosine-compares it against reference sets of harmful and benign prompt representations, and averages the harmful-minus-benign similarity gap across N reference pairs. If this score exceeds threshold τ (default 0), the query is routed through the recovery-adapted model; otherwise it uses the plain task-adapted model.

Preconditions / Affected

Applies to the fine-tuning-as-a-service setting where LoRA fine-tuning is applied to an aligned model on user-submitted data that may be poisoned. Evaluated with poison ratios ρ up to 0.9 (headline numbers use ρ=0.05–0.2), training set size |D_train|=1000 (700 for AlpacaEval).

What to do

No code/commands given on the page itself (repo linked but not reproduced here), but the reported comparative numbers are reproducible reference points: on Llama3.1/SST2, average harmful score drops from 11.54 (plain SFT) to 0.08, versus other defenses in the same test — Lisa 1.70, Antidote 1.50, BDS 1.68, One-shot FT 4.14 — while SLDR's downstream accuracy (92.32 avg) matches undefended SFT (92.39), unlike Lisa (74.17) or Antidote (84.86), which cost substantial accuracy. Across four model architectures on SST2, SLDR keeps harmful score ≤0.20 everywhere, including Mistral-v0.2 where undefended SFT harmful score is 49.40 and every other baseline leaves double-digit (Lisa 12.80) or high single-digit scores.

Caveats

The sensitivity-score probe and layer selection happen on the aligned model before fine-tuning, so it assumes access to the pre-fine-tuned base model to compute κ_l — the page doesn't address cases where only the post-fine-tuned model is available. Reference harmful/benign representation sets are sourced from two other papers' datasets, and the dynamic-routing threshold τ=0 is the default without the page detailing sensitivity to that choice beyond "studied in Section 5.2," which isn't in the fetched text.