Do User-Authored Permission Policies Improve Protection Against AI Agent Overreach?

What it claims

Beyond the headline finding the summary already states, the paper carries a second, technical contribution and a set of design specifics the summary omits. The permission mechanism is built as an MCP proxy that sits between agent and tool server, uses an LLM to classify tool metadata into plain-language consequence categories, and applies user rules before forwarding each tools/call. The four user-facing categories are spend money, send or publish information, delete something, and access private information, derived from three taxonomy dimensions: action type, data sensitivity, and externality.

How it works

POLICY participants set one standing allow/ask/never rule per category before a simulated 18-action day. Actions outside the four categories (5 of them) ran automatically. Crucially, the user study used fixed, researcher-checked tool→category mappings, so live classification errors could not affect the condition comparison — the mapper is evaluated separately. The study adapts AgentDojo's assigned-task vs. additional-action distinction, but here the agent itself initiates overreach rather than an external prompt-injection attacker.

Evidence

• Runtime prompts dropped from 18.0 to 10.9 under POLICY, but total intervention time was not reliably lower once rule-setup time was counted.
• Rule choices: 114 of 140 rules were ask. Of 148 overreach actions executed in POLICY, 133 followed human runtime approval, 15 ran automatically under allow.
• Mapper evaluation used a 538-tool corpus (36 MCP servers) with a 120-tool human-labeled eval set, three annotators (Krippendorff's α=.88 action type; Fleiss' κ=.89 sensitivity/externality). An independent LLM matched human consensus on 95.8% of main-action labels, 85.7% data-sensitivity, 86.6% externality.

Limits

• Not tested against adversarial metadata or prompt injection — the threat model names compromised/misleading tool descriptions but the evaluation explicitly does not cover resistance to them.
• Proxy implementation supports only stdio transport and one upstream server; multi-server and streaming are out of scope.
• Between-subjects design compares the three permission systems as wholes, so no single feature's effect is isolated.
• Participants were non-professional users on everyday tasks (email, travel, purchases), not developers using coding agents; the four categories were not meant to cover every consequence.