LLM Agents for Automated Post-Training via Structured Recipe Optimization

What it claims

SRO automates LLM post-training when no training data is provided. The agent gets a base model, target domains with their benchmarks, a compute environment and a time limit. It must find or generate data and pick the training configuration itself. The objective is the mean over domains of each domain's mean benchmark score. The headline results are only those in the abstract: 41.1 vs 15.4 for the strongest baseline (Qwen3-1.7B, GPT-5.4 as agent) and 53.8 vs 40.8 (Qwen3-4B, DeepSeek-V4-Flash). Both come from joint post-training over three domains and six benchmarks. Code is at https://github.com/RuohLiuq/SRO.

How it works

The paper places SRO between two prior approaches. AutoPipe runs Bayesian optimisation over a fixed config space of base model plus SFT/RL hyperparameters. PostTrainBench-style agents make open-ended edits on each trial.

Recipe-Space Designer

• Schema. A recipe picks one choice for each component:
• per-domain data-source and data-processing components
• one cross-domain composition component
• one shared training-configuration component
• Fixed workflow and skills. A fixed executable workflow handles loading, formatting, training, evaluation, validity checks and compute limits. It also excludes evaluation data from training. The agent therefore never reimplements the pipeline, which the authors say would waste compute and add variance. Supplied "skills" cover:
• dataset discovery
• generation-prompt design
• cross-domain composition
• training configuration
• Choices the agent builds:
• Data sources: datasets it inspects and maps to workflow fields.
• Data processing: two data-generation templates, one for broadening coverage and one for fixing model deficiencies. Generation runs lazily, only when a recipe selects it. Each generated dataset is registered as a new data-source choice, so the space grows during search.
• Composition: mixture ratios and assembly modes under a fixed training-sample budget.
• Training configuration: SFT or GRPO configs.

Multi-Domain Recipe Optimizer (one iteration):

flowchart TD
  H[Trial history: recipe + per-domain scores] --> M[Selective retrieval: Pareto-optimal, poor/regressive trials, choice coverage]
  M --> P[LLM proposes several valid, non-repeated recipes]
  H --> B[Domain-wise surrogates]
  P --> S[Pick max expected hypervolume improvement]
  B --> S
  S --> T[Generate data, train, eval all benchmarks]
  T --> H
  T -->|new generated dataset| R[Recipe space grows]
  R --> P

1. The LLM proposer sees only a compact retrieved summary of past trials.
2. The Bayesian step fits per-domain surrogates on the full history.
3. Pareto-aware acquisition (expected hypervolume improvement) selects exactly one candidate to train.
4. The loop runs until the time limit.
5. It returns the checkpoint with the highest aggregate score for final external evaluation.

Evidence

The fetched text contains only the two abstract numbers above. Their stated setting is three capability domains, six benchmarks, joint post-training. The authors also claim SRO has the top average score in every base-model × agent-LLM combination they tested. The page does not show the full results table.

Limits

• Truncated page. The text stops in the middle of the method section. Experiments, ablations and stated limitations are not in it. Missing details include:
• which domains and benchmarks were used
• which method was the "strongest baseline" (AutoPipe and PostTrainBench are named only in related work)
• time limit, compute budget, trial counts and variance
• Checkpoint selection. The final checkpoint is chosen by aggregate score on the target benchmarks during optimisation. Check in the full paper whether the "external evaluation" uses held-out data. If it does not, the reported scores may be optimistic.
• Scaffolding. Performance depends on the supplied workflow, skills and the two generation templates. Porting SRO means porting all of these.
• Scale tested. Only small Qwen3 targets (1.7B, 4B) appear in the visible text.