Skill-Decoupled On-Policy Expert Distillation for Unified Vision-Language Models

What it claims

An 8B VLM post-trained with SCOPED gets the highest average score across eight multimodal benchmarks among the evaluated models. The authors say it beats larger open-source models and leading closed-source baselines, as well as the backbone and mixed-capability RL. The paper's argument is that multi-teacher on-policy distillation (MOPD) copies experts' token distributions wholesale, including reward-hacking and over-specialized habits. Conditioning each teacher on a reflected "skill" text is meant to filter those out before they reach the student. Code and data are at https://github.com/NEUIR/SCOPED.

How it works

The paper uses four capability groups: fine-grained perception, OCR & chart, symbolic visual reasoning, and multi-image reasoning.

1. Independent RL experts. The training data is split into C capability subsets. Each expert is initialized from the same base policy and optimized on its own subset with a capability-specific reward. Reward design is in Appendix A.3, which is not in the fetched text.
2. Trajectory repository. Each trained expert samples K rollouts per training instance. These are stored with the input and the gold answer.
3. Reward-guided reflection. Rollouts are split at a reward threshold γ into successful and failed cohorts. The expert itself reflects over both cohorts, keeping positive patterns and diagnosing failures. A second prompt then has it synthesize the reflections into one procedural skill per capability. In the paper's example the skill is "compare urban vs rural year by year… do not answer from a single column."
4. Skill-decoupled OPD. The student generates responses unconditioned. At each token, the teacher distribution is computed as the expert conditioned on (x, skill, student prefix). The loss is token-level reverse KL(student ‖ teacher), averaged over response length and over capabilities. The student never sees the skill, so no skill prompt is needed at inference.

flowchart LR
  B[Base policy] --> E1[Perception expert]
  B --> E2[OCR/Chart expert]
  B --> E3[Symbolic expert]
  B --> E4[Multi-image expert]
  E1 & E2 & E3 & E4 --> R{Rollouts: reward ≥ γ?}
  R -->|success| F[Reflect]
  R -->|failure| F
  F --> S[Per-capability skill]
  S -.teacher only.-> T[Skill-conditioned teacher]
  St[Unified student rollout] --> T
  T -->|reverse KL| St

The key difference from plain MOPD is the dotted edge: skills condition the teacher only, never the student.

Evidence

The benchmarks are:

RL data comes from six public datasets, including task-relevant subsets of Orsta-Data-47K.

According to the authors, the ablations show:

• Distilling from experts helps.
• Adding skills helps further on top of that.
• Mixed-capability RL reaches lower training reward than independent RL, aligns weakly with expert directions, and shows recurring gradient conflicts.

Limits

• No results numbers. The fetched text stops at the evaluation-metrics section, so no per-benchmark scores, baseline names, or margins appear. Check the tables before relying on the "beats closed-source" claim.
• Unspecified details. The backbone is not named beyond "8B." The values of K and γ, the reward functions, and the reflection prompts are all deferred to appendices that are not shown.
• Training cost. No compute figure is given. The pipeline needs C separate RL runs, then K-rollout sampling, then reflection, then distillation.
• Skill granularity. There is one skill per capability, not per instance. Its quality depends on the expert's own reflection, and the shown text does not evaluate that separately.