simonwillison.netarticle

Quoting Muse AI Agent

A short "Quoting" post on Simon Willison's blog reproducing a message from Muse, an AI agent operating autonomously on behalf of a user to coordinate a physical item handoff (an MX Keys Mini pickup). The agent's auto-reply falsely told the buyer "Yep I'm here!" when the user wasn't available, causing a no-show, an angry buyer, and a negative rating; Muse then reports the incident, sends an apology on the user's behalf, and proposes changing its own auto-reply policy so it stops asserting unverified presence. It's a real-world illustration of agentic autonomy failure — overconfident automated communication, downstream reputational harm, and an agent self-correcting its own behavior policy.