How Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call Traces

What it is

PrivacyTrace is a benchmark from the Shanghai AI Laboratory (AgentDoG Team, arXiv 2610.41885, 30 October 2026) that measures how much a passive observer can learn about a user from MCP tool-call traces alone, with no access to the conversation or the agent's reasoning. It uses 1,000 synthetic profiles with 17 ground-truth attributes each, 4,000 trajectories and 67,950 tool-call records. Code and data are at https://github.com/AI45Lab/PrivacyTrace.

How it works

The domains are travel, health, shopping and career learning. Profiles are sampled, filtered by an LLM consistency reviewer, and calibrated to 25 population margins (UN DESA, World Bank, UNESCO, ILOSTAT) with iterative proportional fitting. A user simulator holding the profile talks to a Pydantic AI agent that calls real tools on the 25 MCP servers. The extracted trace keeps server, tool name and schema, arguments, returned results or errors, and call order.

The attacker LLMs were MiniMax-M3, DeepSeek-V4-Flash, GPT-5.5 and Gemini-3-Flash. Each had to predict every attribute from a fixed schema, which was compared with a prior-only guess from the same schema.

Preconditions / Affected

The threat model has two semi-honest observers:

• A local observer is a single MCP server that sees only calls routed to it.
• A global observer is a gateway or platform that sees the full trace. It can concatenate sessions if it has a persistent user identity.

The risk therefore falls mainly on gateways and multi-server deployments that link sessions, and on any server that logs full arguments and results.

What to do

The page states no mitigations. It only calls for privacy-aware mechanisms for agent-tool interactions. The measurements below show where to look first:

• Channels: Arguments add 13.18 percentage points of attack success over metadata and call sequences alone. Returned results add 10.01 points. Tool names and ordering matter least.
• Linking: A linked four-domain view reaches 32.77% success, above the 26.25% average across all views. Limiting retention and identity linkage reduces this exposure.
• Residual inference: Attributes inferred from the trace improve guesses for the remaining attributes by 3.30 points over prior-only guessing.

Caveats

• Average success is 26.25% against a 17.97% prior baseline, a gain of about 8 points. Absolute accuracy is low.
• Highly specific identifiers are hard to recover, with only 35 correct among 16,000 identifier slots.
• A two-stage attack (select trace-supported attributes, then complete the rest) performs about the same as one-shot inference.
• The data is synthetic and the tasks are generated from the profiles, so real-user rates may differ. The page does not discuss this.
• The fetched text ends partway through the attack design section. Per-attribute, per-domain and per-model results, and any defence evaluation, are not covered here.