AREX-2: Advancing Self-Improving Agents through Long-Horizon Reflective Tasks

What it claims

AREX-2 is a fine-tune of Qwen3.8-27B. It is trained by imitation on long, multi-round improvement trajectories from ML engineering and algorithmic programming. The paper's thesis is that the retry loop usually lives in the agent scaffold, and these trajectories move it into the model. It frames self-improvement as total gain ≈ r̄·T*:

• r̄ is the mean gain per productive round. The paper attributes this to reflection.
• T\* is the number of rounds that stay productive. The paper attributes this to long-horizon execution.

A bigger round budget helps only up to T*.

How it works

flowchart TD
  S[GitHub repos / online judges] --> T[Teacher builds task + scorer + sandbox]
  T --> G{Reference scores AND baseline well below?}
  G -- no --> X[Discard env]
  G -- yes --> A[Teacher agent: hours, hundreds of tool calls, skills, feedback each round]
  A --> K{Final best score ≥ threshold AND rules/format OK?}
  K -- no --> Y[Discard trajectory]
  K -- yes --> L[SFT with loss on progress steps only]

Environments.

• For repos, the scorer computes the repo's own metric on a held-out split the agent cannot access.
• For judge problems, the score is graded wherever possible, such as the fraction of hidden tests passed or heuristic quality versus the best known solution. Binary verdicts are avoided.
• An environment is kept only if the reference solution runs and a naive baseline scores well below it, so there is room to improve.

Trajectory generation.

• The agent is told its budget, which is far larger than one attempt needs.
• It gathers "operational knowledge" by reading source code, searching papers and libraries, and running small experiments.
• It is given skills: compact context documents. General skills come from ML repos and common practice. Task-specific skills come from searches, with material about the task itself excluded.
• Rounds that lower the score are kept in the trajectory.

Selection. Whole trajectories are filtered on final outcome only, never on individual steps. Trajectories that reconstruct held-out answers or expose credentials are rejected.

Loss masking.

• Failures stay in the context as conditioning.
• Loss applies only to the decisions that make progress: diagnosing, repairing, changing strategy, running the next experiment, submitting an improvement.
• Repeated polling, no-op calls, near-duplicate turns, system messages, environment reports and retrieved documents get no loss.

Data mix. The new trajectories are combined with the prior AREX deep-research data, unchanged.

Evidence

Figure 1 compares AREX-2 against selected closed and open models. The test-time round budget and scaffold for these numbers are not in the fetched text.

The clear wins are on MLE-bench Lite and GAIA, against the earlier AREX models. BrowseComp and HLE margins are under one point.

How to use it

• Code: https://github.com/VectorSpaceLab/AREX-2
• Weights: https://huggingface.co/collections/BAAI/arex-2

The trajectories were generated with skills in context, and the paper says skills "remain available to it at test time". Expect the model to perform best inside a scaffold that supplies such documents, a round budget and per-round scored feedback.

Limits

• Text cut off. The fetched text stops at the start of the Experiments section. The rounds-scaling curve, ablations and evaluation settings were not available.
• Unstated details. The licence, teacher model, dataset size and training compute are not stated in what was fetched.
• Transfer evidence is confounded. The authors call the deep-research gains transfer because the deep-research data is unchanged. However, the base model also changed: earlier AREX models are listed at 122B and 4B. No same-base, old-recipe control appears in the fetched text.
• Required environment. The method assumes graded, verifiable scorers. Using it on your own tasks means building environments that pass both admission checks.