Foundations of Proactive Agents: Principles, Technical Layers, and Proactivity-Gym

What it claims

Proactive agents should be scored on three separate axes, not one. These are Task Capability (TC), Temporal Allocation (TA) and Trust (TR). Existing benchmarks merge them into a single "task/intervention success" score, which hides failures. The paper evaluates 23 model–harness configurations on its PROACTIVITY-GYM testbed:

• TC correlates only weakly with TA (r = 0.31) and with intervention-depth alignment, TR-D (r = 0.34).
• Agents struggle most to defer competing work.
• LLM-judge trust scores (TR-J) barely react to intervention-depth misalignment. They mostly follow TC. If you use an LLM judge for trust, you are largely measuring task quality again.

How it works

• Objective: a weighted sum, max over designs A of λ_TC·S_TC + λ_TA·S_TA + λ_TR·S_TR, where the λ values are positive and sum to 1.
• Interaction time vs sleep time (after Lin et al., 2025):
• Interaction time is an active workflow phase, where proactive work competes with the user for resources such as GPU capacity or token budget.
• Sleep time is an inactive phase between active ones, free for background work.
• Short idle gaps inside an active phase still count as interaction time.
• Intervention depth comes from the IP continuum, via Kraus et al. It has three levels:
• prepare: gather material, nothing shown or applied
• suggest: present for review
• execute: act without confirmation
• Trust proxy: depth is used as an observable stand-in for trust. A user who stops allowing autonomous edits is treated as signalling lower trust.
• Trust constructs: five, from McAllister and Madsen & Gregor. They are understandability, technical competence, reliability, personal attachment and faith.
• Testbed: 10 hand-crafted multi-day scenarios, with 3 user personas each. It has a simulated clock, stateful tool environments and persona-conditioned simulated user feedback, so later state depends on what the agent did earlier.

The paper's worked example (Fig. 2) shows where TA and TR decide the outcome:

flowchart TD
  A[Agent spots missing ablations + summary] --> B{GPU and user busy?}
  B -- yes --> C[Defer to sleep time]
  B -- no --> D[Run now]
  C --> E{User's depth preference}
  E -- appendix: delegated --> F[Execute: update appendix]
  E -- main text: approval --> G[Suggest: hold summary for review]
  F --> H[Handoff before deadline]
  G --> H

Evidence

Benchmark (23 configurations): the correlation and LLM-judge findings are listed above. The fetched text gives no per-model scores or model names.

Human study (30 participants, scenario-based):

• Participants often rejected help that competed with their ongoing tasks. They accepted the same help when it was scheduled for sleep time, even when the output needed correction.
• They favoured deferral even when acting immediately caused no resource or deadline conflict, to protect their current focus.
• A single intervention-depth misalignment caused a sharp drop in trust, even when the task outcome was correct.
• Returning to aligned behaviour did not necessarily restore the earlier trust level.

Implications for builders:

• Default to deferring proactive output rather than interrupting.
• Treat an autonomy overstep as costly and hard to recover from.
• Do not validate trust with an LLM judge alone.

How to use it

The project page is https://harryoh99.github.io/foundations_of_proactive_agents/. The fetched text includes no code, install instructions, release status or licence for PROACTIVITY-GYM.

Limits

• Truncated page: the text cuts off at Sec. 4.2 (situation and system modeling). Not visible:
• the testbed's metric definitions for S_TC, S_TA and S_TR
• per-model results
• the human-study protocol
• Small, hand-built benchmark: 10 scenarios with 3 personas each.
• Simulated users: feedback comes from persona-conditioned simulations, not real users.
• Study design: the human study is scenario-based, with 30 participants.
• Unstated λ weights: the text does not say how the weights in the objective are chosen.
• Preprint: this is v1, dated 29 Sep 2026.