Skip to content

Back to the catalog

ANT: A Multi-Granularity Network Traffic Dataset and Benchmark for Agents Behavior Auditing
arxiv.orgpaper

ANT: A Multi-Granularity Network Traffic Dataset and Benchmark for Agents Behavior Auditing

ANT (Agent Network Traffic) is a dataset and benchmark for auditing LLM agent behavior from network traffic without inspecting private user content, covering 3,114 execution episodes across 20 tasks and five scenarios, with 276,417 bidirectional flows and 40,049 behavior-primitive segments organized into 47 macro groups. The authors benchmark 13 traffic-analysis baselines on three tasks — agent risk identification, scenario recognition, and behavior primitive classification — finding that methods struggle when malicious workflows mimic benign tasks and when scenarios share similar traffic patterns, while primitive classification works better for frequent, distinctive macro groups than for rare or semantically similar ones.

TL;DR A new dataset and benchmark for detecting risky LLM agent behavior from network traffic alone, exposing where current traffic-analysis methods fail to catch malicious agent workflows disguised as benign tasks.
Open original ↗
Also seen from another angle🧠 AI →
Document

What it is

ANT (Agent Network Traffic) is a dataset and benchmark from Tsinghua University for inferring LLM agent behavior purely from network traffic metadata (packet sizes, timing, direction) without payload inspection. Data and code are at the anonymous repo anonymous.4open.science/r/ant-main-suite-7BC0. Collection cost $2,176.50 across geographically distributed cloud servers, producing 104.07 GB of traffic over 659 distinct task instances.

How it works

Agents (five configurations: Claude Code+DeepSeek domestic/overseas, Qwen, Codex, Claude Code+Sonnet, Claude Code+MiMo) run tasks in isolated virtual network namespaces while tcpdump captures PCAPs and session logs record model calls/tool invocations with timestamps. Boundaries from session logs segment the packet stream into "behavior primitives" (e.g., WebSearch, WebFetch); DeepSeek-v4-pro labels each segment from aligned log evidence, and 3,295 distinct primitive labels are embedded with Jina Embeddings v5 and clustered via spherical K-means (K=47, chosen by cosine silhouette) into macro groups.

Preconditions / Affected

Relevant to teams building traffic-only (non-DPI) detection of agent activity, e.g., for shadow-AI governance. The cybersecurity-misuse scenario uses three tasks derived from Meta's PurpleLlama Cybersecurity Benchmarks and OWASP Juice Shop, with attack code generation drawn from PurpleLlama's mutated prompts (no added jailbreak/obfuscation) and refusal-only episodes excluded.

What to do

Per-scenario volumes for anyone selecting a subset to benchmark against: Computational Artifact Engineering (516 episodes, 33.41 GB), General QA (1,014 episodes, 16.55 GB), Evidence Assessment (853 episodes, 8.47 GB), Interactive App Operation (315 episodes, 14.77 GB), Cybersecurity Misuse (416 episodes, 30.87 GB). Reported baseline results to calibrate expectations: all 13 traffic-analysis baselines exceed 0.900 accuracy on risk identification, but 7 of 13 have malicious-class recall below 0.500 — meaning high aggregate accuracy masks poor detection of actual misuse. Best scenario-recognition baseline reaches only 0.727 accuracy. Each scenario's dominant macro group is also given, e.g. "search" accounts for 33.46% of General QA primitives and 42.75% of Evidence Assessment primitives, while "code creation" dominates both Computational Artifact Engineering (16.39%) and Interactive App Operation (12.01%) — useful as a sanity check when reproducing primitive classification.

Caveats

The paper reports that cross-task cohesion within a scenario is modest (15.46% smaller MMD distance, 14.79% smaller Hellinger distance versus cross-scenario pairs), so scenario-level traffic signatures are statistically real but weak. The malicious-class recall figures above are averaged in the sense that "seven of thirteen" fall under 0.500 — the paper doesn't name which methods in the text extracted here.

Organisations named here

  • ANT (Agent Network Traffic)

Technologies named here

  • LLM

Related in AI Security