ANT: A Multi-Granularity Network Traffic Dataset and Benchmark for Agents Behavior Auditing

What it claims

ANT pairs packet captures of LLM agent runs with labels at three levels: episode risk (benign vs. cybersecurity misuse), scenario, and per-stage behavior primitive. The paper uses it to ask how much of an agent's task can be inferred from encrypted traffic alone. The headline result: all 13 baselines exceed 0.900 accuracy on risk identification, but seven of them have malicious-class recall below 0.500. Accuracy is therefore a misleading metric here. The best scenario-recognition accuracy is 0.727.

How it works

Collection ran on geographically distributed cloud servers. Each task instance ran in its own network namespace. tcpdump captured all packets while the agent harness logged model calls, tool calls, outputs and session state with timestamps.

flowchart LR
  A[Task prompt + workspace] --> B[Agent run]
  B --> C[Session logs]
  B --> D[tcpdump PCAP]
  C --> E[Bin packets at primitive start times]
  D --> E
  E --> F[LLM labels each segment]
  F --> G[Embed + cluster labels into 47 groups]

• Segmentation. Packets are sorted by arrival time and cut at each primitive's start time from the logs. A flow that spans a boundary is split packet by packet. Traffic before the first primitive gets a synthetic prelude label.
• Labeling. DeepSeek-v4-pro produces a free-text primitive from the aligned log evidence. The 3,295 distinct labels were embedded with Jina Embeddings v5 and clustered with spherical K-means. K=47 was chosen by cosine silhouette, seed stability and minimum group size. Labels were refined with AI assistance until a 2,000-sample spot check found "no substantial errors."
• Agent configurations:
• Claude Code + DeepSeek, overseas (20.6%) and domestic (20.3%)
• Qwen (20.1%)
• Codex (15.5%)
• Claude Code + Sonnet (12.3%)
• Claude Code + MiMo (11.2%)
• Tasks.
• 17 benign tasks are derived from existing benchmarks; one cited example is a GAIA question.
• The 3 misuse tasks come from Meta's PurpleLlama Cybersecurity Benchmarks and OWASP Juice Shop.
• There are 659 distinct task instances in total.

Evidence

• Size and cost. The total is 104.07 GB, and collection cost US$2,176.50.
• Scenario structure in the labels. Distinct tasks within the same scenario have 15.46% smaller primitive-embedding MMD and 14.79% smaller Hellinger distance than cross-scenario pairs (Holm-adjusted p=0.0006 and p=0.0050). Primitive ordering adds roughly 5–6 percentage points of cohesion beyond composition alone (p=0.0050).
• Why traffic carries signal. In Claude Code, WebSearch is delegated to the model backend, so its traffic is mostly backend communication. WebFetch adds DNS lookups and content downloads.
• Primitive classification. Baselines do better on frequent groups and on groups with distinctive traffic patterns.

How to use it

Data and code are at an anonymized review link: https://anonymous.4open.science/r/ant-main-suite-7BC0/. The page states no licence. Splits and per-group counts are in Appendix B.2.

Limits

• Truncated page. The fetched text ends before the benchmark tables. The 13 baselines are not named, and per-method numbers are not available here.
• Class imbalance. Misuse is only 416 of 3,114 episodes. Report malicious recall, not accuracy.
• Rare and similar groups. Primitive classification struggles with rare groups and semantically similar ones.
• Label quality. Labels are LLM-generated and AI-assisted. Validation is a single spot check with no reported error rate.
• Narrow misuse coverage.
• It comes from public PurpleLlama prompts with no jailbreak or obfuscation.
• Refusal-only episodes were excluded.
• Attacks ran inside isolated Docker environments, and state-changing benign tasks hit locally simulated services.
• Real-world evasive misuse is not represented.
• Client-side capture only. All traffic was captured on the agent's own host. Coverage is limited to the six harness/backend combinations listed above.