Skip to content

Back to the catalog

Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation
arxiv.orgpaper

Rethinking LLM Fine-Tuning via Weight Space Reparameterization: Preserving Safety during Downstream Adaptation

WSR-Tune is a fine-tuning framework that reparameterizes LLM weight matrices with a safety-conditioned basis, so safety-relevant information concentrates in a small set of directions. Those safety-critical directions are frozen, and the remaining complementary directions are updated for the downstream task. The authors report a better trade-off between downstream performance and safety retention. The page gives no benchmark names or numbers.

TL;DR Proposes WSR-Tune, which freezes safety-critical directions in a reparameterized weight space while fine-tuning the rest to keep LLM safety.
Open original ↗
Document

What it claims

WSR-Tune (Yonsei University, NeurIPS 2026, arXiv October 2026) argues that freezing safety-relevant parameters fails because safety is spread across many coordinates of the original weights. The fix is to change the basis first, then freeze. On safety-tuned Llama-2-7B, freezing 10% of coefficients in the reparameterized space gives a harmfulness score of 14.67 after downstream fine-tuning. Freezing 50% of the original parameters gives 18.43.

How it works

The method is applied per selected layer W_l:

  1. Run a safety corpus D_safe through the model. D_safe holds harmful prompts paired with safe refusal responses. Collect the input activations to W_l for every valid token.
  2. Compute the activation covariance H Hᵀ and take its SVD to get an orthonormal basis U_l. On Llama-2-7B-Chat the singular values show a clear low-rank structure (Figure 2), which the authors use to justify this basis.
  3. Reparameterize as W̃_l = V_lᵀ W_l U_l, with V_l = I. Each coefficient corresponds to a basis matrix φ_ij = v_i u_jᵀ. The basis is built once and stays fixed during fine-tuning.
  4. Score each coefficient by the summed absolute gradient of the safety loss on D_safe with respect to W̃_l. Freeze the highest-scoring coefficients and train the rest on the downstream task.

The authors state that existing adaptation methods can run in the reparameterized space without changes to their update rules. Appendix C reportedly shows robustness to other choices of V_l.

Evidence

Table 1 uses safety-tuned Llama-2-7B. The safety score (lower is better) is averaged over Direct, AutoDAN, PAIR and PAP attacks after downstream fine-tuning:

SettingSafety↓
Full FT (0% frozen)53.84
FT, 10% original params frozen33.95
FT, 30% frozen21.44
FT, 50% frozen18.43
WSR-Tune, 10% frozen14.67

The page names related baselines but gives no head-to-head numbers against them. These are SafeInstr, Lisa, SEAL, SafeGrad, SaLoRA, SN-Tune, AsFT, SafeLoRA, SafeDelta and RESTA.

How to use it

Code is at https://github.com/min-seong-kim/WSR-Tune. Adoption requires a safety dataset of harmful prompts with refusal responses, which is used both to build the basis and to score coefficients. The fetched text does not give that dataset's name or size.

Limits

The fetched text stops partway through Section 3.3. It does not include the freezing-selection rule beyond gradient scoring, the full experiments, or a limitations section. The downstream task behind Table 1 is not named, and Table 1 reports no downstream accuracy. The trade-off with task performance therefore cannot be checked from this page. The only model shown is Llama-2-7B (and its Chat variant for the spectrum analysis). The cost of activation collection and per-layer SVD is not stated.

Organisations named here

Technologies named here

Techniques named here

  • Weight Space Reparameterization

Related in research