Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use

What it claims

The paper puts concrete numbers and mechanisms behind the summary's claim of degradation. Backbone is Qwen2.5-7B-Instruct, trained via full-parameter SFT and via RL (GRPO with sparse answer/format rewards only, no detailed failure guidance, to match SFT). The sandbox dataset is 6,050 training / 880 evaluation samples over geographical POI queries and calculation tasks, with strict separation so no test-set variation pattern appears in training. The headline diagnostic split: SFT and RL fail through distinct modes — SFT via trajectory overfitting and brittle symbolic anchoring (memorizing identifiers/template tokens), RL via boundary blindness from a teleological reward bias (it keeps trying to succeed where it should refuse). This is a conference paper (ICML 2026, PMLR); the fetched text cuts off partway through Tier 1, so the full results tables are not visible here.

How it works

Shifts are cast as cascading perturbations along the agent-environment loop: a perturbation at step t alters observation oₜ, which enters history and conditions all later decisions, so errors compound — unlike passive-model perturbations where each input is independent. The four shift dimensions are made concrete:

• ∆Q (query): instruction blurring (colloquialisms, implicit references) and information redundancy (core instruction buried in irrelevant context).
• ∆A (action): surface drift (rename identifiers, same function), semantic conflict (same name, altered behavior), structural reconfiguration (merge/split/invert tools); also schema shifts like renaming the arguments key to args/parameters.
• ∆O (observation): null values, error states, corrective instructions — the agent must separate informative anomalies (adjust policy) from terminal failures (refuse).
• ∆D (domain): jointly changes Q, A, O while preserving a latent problem-solving structure the agent must transfer.

PAFT injects controlled observation anomalies and symbolic noise into SFT training trajectories to break shallow pattern memorization.

Evidence

Four metrics are defined, which is the actionable part the summary omits:

• Tool Error Rate (TER) = N_error/N_total — robustness.
• Active Exploration Score (AES) = e^−(n_old−1), where n_old is repeated invalid attempts; 1.0 = immediate adaptation — adaptability against "cognitive inertia."
• Average Tool Chain Length (ATCL) — efficiency on solvable tasks.
• Refusal Rate (RR) = N_refusal/N_unsolvable — boundary awareness on unsolvable cases.

The closed-setting baseline (Figure 1) shows both paradigms converging to near-perfect success before shifts are applied, establishing that later degradation is attributable to the shifts.

How to use it

Code is promised at https://github.com/LAMDA-NeSy/OpenAgent. If you train tool-use agents, the metric set (especially RR and AES) is directly reusable for testing whether your agent adapts vs. loops vs. refuses under shift, independent of the paper's specific tasks.

Limits

Everything runs in a synthetic sandbox on a single backbone (Qwen2.5-7B-Instruct) over POI/calculation tasks — the authors chose this to isolate variables from real-API noise, so open-world API instability is explicitly not tested. The fetched text ends inside Tier 1 (Perception), so the actual per-tier degradation figures and PAFT's quantitative gains are not in this excerpt — verify them against the paper before citing magnitudes. PAFT targets SFT only; no analogous fix for RL's boundary blindness is described here.