How Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call Traces

What it claims

PrivacyTrace (Shanghai AI Laboratory, arXiv October 2026) asks what a semi-honest MCP server or gateway can infer about a user without seeing the conversation. It sees only the tool names, schemas, arguments, results, errors and call order it legitimately receives. Four LLM attackers (MiniMax-M3, DeepSeek-V4-Flash, GPT-5.5, Gemini-3-Flash) reach 26.25% average attribute accuracy, against 17.97% from prior-based guessing. Linking a user's sessions across domains raises this to 32.77%.

How it works

The threat model separates two observers:

• A local observer is one MCP server and sees only calls routed to its endpoint.
• A global observer is a gateway or platform and sees the full task trace. With a persistent user identity, it can concatenate traces across sessions.

The experiments compare three views: one server's calls, one complete task trace, and four linked domain traces.

flowchart LR
  U[User + agent dialogue] -->|not visible| B{Privacy boundary}
  B --> G[MCP gateway: global observer]
  G --> S1[Server: local observer]
  G --> S2[Server: local observer]
  G -->|link sessions by user ID| L[Cross-domain profile]

Synthetic profiles are built in three steps:

1. Candidates are sampled from a mixture of target-informed and uniform proposals.
2. An LLM consistency reviewer filters out incoherent candidates.
3. Iterative proportional fitting weights the pool to 25 low-dimensional margins drawn mainly from UN DESA, World Bank, UNESCO and ILOSTAT aggregates.

Each of the 1,000 profiles has 17 ground-truth attributes. For each profile, a planner writes one task in each of four domains: travel, health, shopping and career learning. A simulated user then drives a Pydantic AI agent that has the full domain tool catalog. The planner's hidden tool plan is used only for auditing and does not constrain the agent.

Two attacks are evaluated:

• One-shot predicts all attributes from the trace in a single pass.
• Two-stage first selects the attributes the trace supports and proposes their values. A second model then fills in the remaining attributes from that partial profile alone, without the trace.

Evidence

• Channels: Tool arguments add 13.18 percentage points of attack success rate over metadata and call sequence alone. Returned results add 10.01 points.
• Visibility: Linked four-domain traces reach 32.77%, above single-server views.
• Identifiers: Specific identifiers are rarely recovered: 35 correct predictions out of 16,000 identifier slots.
• Chaining: Attributes inferred from the trace improve accuracy on the unresolved attributes by 3.30 points over prior-based guessing on the same targets.
• Attack design: Two-stage inference performs comparably to one-shot.

The dataset has 4,000 trajectories and 67,950 tool-call records from real MCP tools on 25 servers.

How to use it

Code and data are at https://github.com/AI45Lab/PrivacyTrace. For anyone running an MCP gateway or server, the channel results point to arguments and responses as the parts of a log to minimise or protect. Metadata and call order alone leak less.

Limits

Profiles and user turns are synthetic and LLM-simulated. They are calibrated only to the 25 fitted margins, not to real user behaviour. Results cover four domains and four attacker models. The fetched text stops at the start of Section 4.2, so per-model, per-domain and per-attribute tables, and any defenses, were not available for review. The paper evaluates no mitigation in the portion shown.