LESSER: Post-Training Data Selection with Output-Layer Gradients

What it claims

LESSER (Last-layer Error Signals for Scalable Example Ranking) replaces the full-parameter gradients in existing gradient-based selectors with output-layer gradients. The selection rule stays the same. The authors apply it to three selectors:

• LESS for SFT data selection.
• GradAlign for online RL problem selection.
• GRACE for choosing a distillation teacher.

The headline numbers are FLOP reductions of 9.7× for SFT and 3.0× for RL. The wall-clock savings are larger:

• 17.8× less feature-extraction time in SFT.
• 3.5× less added scoring time per selection round in RL.
• 3.9–5.4× less feature-extraction time for distillation teacher selection.

How it works

The SFT loss and GradAlign's RL scoring loss share one form: L = −Σ_t α_t log π(y_t | x, y_<t). In SFT, the weights α_t mark the supervised positions. In RL, they also fold in advantages and token weighting. The weights are treated as fixed when differentiating.

The paper's Proposition 1 handles the readout: it is treated as a separate copy W_out of the vocabulary projection W. Its gradient is then

g_p = Σ_t α_t (softmax(W h_t) − e_{y_t}) h_tᵀ

Both h_t and softmax(W h_t) come out of the forward pass, so no backpropagation is needed. The feature is vec(g_p), randomly projected to a lower dimension with a two-sided projection described in Appendix B.2. It is compared to query features by similarity, and the original selection rule runs unchanged:

• SFT: a round-robin pass over the queries to pick k examples.
• RL: aggregate scores across queries, take the top-k problems, update the policy, and re-select each round.

flowchart LR
  P[Pool + queries] --> F{Feature}
  F -->|LESS / GradAlign| B[Full backward pass]
  F -->|LESSER| O[Output-layer grad, forward only]
  B --> S[Similarity scores]
  O --> S
  S --> R[Unchanged selection rule]
  R --> T[Train on subset]
  T -->|RL: next round| P

Evidence

SFT setup. The candidate pool is 197,196 Tulu V2 examples. Query and test tasks are TyDiQA, MMLU-Pro, GSM8K, Codex and BBH. Subset sizes are k = 1k, 5k and 10k. Each subset is fully fine-tuned with three seeds on four models: Llama-2-7B, Llama-3.2-3B, Qwen3-4B-Base and OLMo3-7B.

SFT results.

• Across 60 model–task–budget cells, LESSER's mean absolute score gap is 1.3 points against LESS, 2.2 against RDS+ and 2.6 against Random.
• On Llama-2-7B:
• LESSER roughly equals LESS on GSM8K and MMLU-Pro.
• LESS is stronger on TyDiQA and BBH.
• LESSER is stronger on Codex.
• RDS+ is also competitive on several benchmarks.

Batch-level alignment. The selected batches are compared to LESS-selected batches using centered gradient cosine similarity, measured at the base model:

Similarity-bin diagnostic. The pool is split into similarity deciles, and a model is fine-tuned on the top 500 examples of each bin. On Llama-2-7B, LESSER similarity predicts query loss with a mean Spearman correlation of 0.86, which the paper describes as close to LESS.

RL. In the first round, LESSER's selected problems have a Jaccard overlap of 0.53 with GradAlign's, against 0.075 for random. Both reach the same final test accuracy.

Distillation. In three settings, each with a pool of 10–14 teachers, LESSER picks the same top teacher as unmodified GRACE.

How to use it

The page gives no code, repository or install instructions. To adopt it:

• Implement the closed-form g_p above inside your existing LESS, GradAlign or GRACE pipeline.
• Project the feature down if storage or similarity cost requires it.
• Feature extraction needs only the base model's forward pass. LESSER uses no warmup training and no LoRA checkpoints.

Limits

• The SFT comparison is not feature-only. LESS uses Adam-preconditioned LoRA gradients at four warmup checkpoints, while LESSER uses only the base model. The pipelines therefore differ in more than the feature.
• Baselines are borrowed. SFT baseline numbers come from Nayak et al. (2026), except MMLU-Pro, which the authors reran.
• Per-example rankings disagree. Output-layer and full-gradient rankings differ on individual examples. The method relies on the selected batches aligning, so it should not be used as a per-example influence or attribution score.
• Per-task gaps exist. LESS remains ahead on TyDiQA and BBH for Llama-2-7B.
• Narrow RL evidence. The RL result is a same-final-accuracy claim; the visible text gives no task or model details for it.
• Preprint status. The paper is under review.
• Truncated page. The fetched text cuts off partway through the SFT findings. The RL and distillation experimental details, the remaining results and any limitations section the authors wrote were not visible.