ParetoSaddler: Pareto-Guided Optimization of Agent Harnesses for Quality and Cost

What it claims

ParetoSaddler is a harness optimizer for LLM agents. It tunes prompts, tools and control logic for both quality and cost without changing model weights. The authors argue that applying the final deployment preference at every intermediate step rejects candidates too early, even though those candidates could lead to later gains. ParetoSaddler therefore uses one rule to promote intermediate candidates and a separate step to pick the harness to deploy. The Pareto-selected harnesses reach higher quality at lower cost than quality-only harness optimization. That baseline is AutoSaddler (labelled AS-Quality), the authors' earlier method.

How it works

The search is an iterative parent-to-candidate loop.

flowchart TD
    P[Parent harness] --> O[Optimizer edits harness]
    O --> C[Candidate]
    C --> R[Repeated runs: parent vs candidate]
    R --> B{Bootstrap Pareto support:<br/>credible gain in quality OR cost?}
    B -- yes --> D[Promote: broader dev evaluation]
    D --> Pool[Parent pool]
    B -- no --> X[Discard]
    Pool --> P
    Pool --> F[Final deployment selection]

A candidate is promoted if it shows credible progress on either axis, quality or cost, across repeated executions. It does not need to win under the deployment preference. Bayesian bootstrap screening drops candidates whose gains are weakly supported across runs. Promoted candidates get a wider development evaluation and stay available as parents for later steps.

The authors contrast this with GEPA in the page comments. GEPA applies Pareto selection across task instances to keep candidates diverse. ParetoSaddler applies it across two competing objectives. GEPA's original work also mainly evolves prompts, while ParetoSaddler targets the whole harness.

Evidence

Evaluation uses ClawMark and GAIA2 with domain-disjoint splits. On ClawMark, quality rises from 70.86% to 81.35% and cost falls from $1.17 to $1.01 per task. On GAIA2, quality rises from 56.22% to 62.89% and cost falls from $0.78 to $0.45 per task. The authors' analysis reports that Pareto-aware search keeps candidates that the deployment preference would discard, and that some of those candidates later support improvements.

How to use it

Code is in the feat/paretosaddler branch of https://github.com/microsoft/AutoSaddler. It is a feature branch, not a separate repository. The paper can be pulled into an agent with the Hugging Face CLI:

hf papers read 2611.02079

curl -LsSf https://hf.co/cli/install.sh | bash

Limits

The page carries only the abstract and author comments. It does not name the base models or the optimizer LLM, and it does not give the number of repeated runs or the cost of the search itself. There is no head-to-head comparison with GEPA. The authors say a fair comparison would require aligning optimization scope and objectives, and they point to the AutoSaddler paper for a GEPA comparison against the quality-only baseline. No models, datasets or Spaces link to the paper, and the page states no licence.