How to train your model organism

What it claims

Organisms trained only to install a behaviour suffer damage that the target-behaviour rate does not show, and that damage changes interpretability conclusions. Two examples hold the behaviour rate fixed:

MT-Bench is normalised to the base model. The failure modes behind the low scores are specific:

• The released Pando organism was trained to emit a single yes/no token. When prefilled with "yes, because", it terminates immediately with no rationale.
• The degraded Lottery organism repeats its first sentence six times on an ordinary story prompt.

The paper gives three training recommendations:

1. Prefer DPO to SFT.
2. Do not arbitrarily mix in chat data, despite Minder et al.'s earlier advice.
3. Merge the finetuned checkpoint back toward the base model.

How it works

Validation metrics. Each is normalised so that 1.0 means "indistinguishable from M₀":

• Behaviour. Target rate on the trigger distribution, plus a rate on a counterfactual distribution with the trigger removed. The second checks that the behaviour is trigger-conditional rather than emitted unconditionally.
• Capability. MMLU accuracy, in-domain benchmark accuracy where relevant, and MT-Bench via an LLM judge.
• CoT naturalness. gpt-5.4-mini, given few-shot examples, classifies GSM8K CoTs as coming from the organism or the base model. It uses 100 balanced held-out questions where both models answered correctly. Chance (~50%) counts as natural.
• Activation naturalness. Take the mean residual-stream difference at the middle layer, over 10,000 FineWeb passages and the first 5 token positions. Decode it with Patchscopes (k=20 tokens per position). The score is the fraction of tokens that gpt-5.4-mini judges relevant to the finetuning objective.

Training.

• All recipes use LoRA, with loss on completion tokens only.
• Chat-mixing variants use OLMo 3's Dolci Instruct SFT or Dolci Instruct DPO subsets at a 1:1 ratio. Tool-use, code and multilingual data are excluded.

Merging (adapted from SACPO). Train a target-behaviour checkpoint π_tgt, then interpolate:

W = (1−q)·W_M0 + q·W_πtgt

M₀ serves as the constraint-satisfying endpoint, so no separate constraint-realignment step is needed. For a LoRA adapter, this is equivalent to scaling lora_alpha by q while leaving the trained matrices unchanged. The tuning cost is therefore one knob after training. The method is justified for DPO and KTO. For SFT it applies only if a reverse-KL penalty to M₀ is added (proof in the paper's Appendix C).

Evidence

On Pando car-purchase "no-exp" organisms, the paper compares the released SFT organisms with a DPO retrain. The base model is gemma-2-2b-it, results are pooled over decision-tree depths 1–4 (80 models), and scores are normalised to that base:

Activation naturalness was barely affected. The damage shows up mainly in chat quality and CoT.

The paper's comparison table reports that, among 11 prior organism papers, none covers all five axes. Most report only the target-behaviour rate plus at most one other axis.

How to use it

• Code: https://github.com/Rice-wxl/multi_objective_mo
• Clinical organisms (demographic-bias shortcuts on patient gender, age and race), with data and checkpoints: https://huggingface.co/multi-objective-mo

The page states no licence, install commands or base model for the clinical suite.

Limits

• The fetched text stops partway through Section 4. The clinical-suite results, the DPO-vs-SFT comparison on that suite, the investigator-agent audit, and the reasoning behind the advice against chat mixing appear only as abstract-level claims.
• The naturalness metrics depend on an LLM judge (gpt-5.4-mini) and on a modest 100-question CoT set.
• The paper assumes that "realistic" means "close to M₀". That assumption is the basis of both the metrics and the merge endpoint.
• The q selection procedure and the constraint floors used are not shown in the fetched text.
• This is a v1 preprint.