Skip to content

Back to the catalog

SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs
arxiv.orgpaper

SparLeak: Privacy Leakage from Sparse Attention in LLM Inference on Shared GPUs

SparLeak is a side-channel attack exploiting a newly identified GPU micro-architectural leak, Sparsity-Induced Memory Access (SIMA), caused by input-dependent key-value cache access patterns in sparse attention mechanisms used for long-context LLM inference on shared GPUs. The attack recovers prefill-phase query attributes and reconstructs decoding-phase LLM responses by profiling token-level sparsity patterns from page-level memory observations, achieving 90.9% attribute-inference and 87.3% response-reconstruction success across three LLM architectures, three sparse-attention mechanisms, and three privacy-sensitive datasets. The authors release anonymized SIMA traces, trained attack models, and evaluation scripts/artifacts.

TL;DR A GPU side-channel attack (SparLeak) exploits sparse-attention memory access patterns to steal private LLM query/response content on shared GPUs.
Open original ↗
Document

What it is

SpaRLeaK targets a new GPU micro-architectural side channel the authors call Sparsity-Induced Memory Access (SIMA). It arises because sparse attention (used in serving engines like vLLM and SGLang to speed up long-context inference) makes input-dependent decisions about which KV-cache pages to fetch from HBM, and those decisions leak through shared GPU cache/TLB state on multi-tenant GPUs.

How it works

SpaRLeaK uses two distinct extraction primitives tied to the two inference phases: an L2-cache-based Invalidate+Compare primitive to recover cumulative SIMA traces during the parallel prefill phase, and a TLB-based Evict+Reload primitive to extract step-wise traces during the sequential decoding phase. Because KV tensors are managed at page granularity (64KB–2MB) while a token's KV state is only ~10-30KB, each observed page access aggregates multiple tokens — so the attack performs trace reconstruction, exploiting spatial locality and statistical regularities in sparsity to approximate token-level profiles rather than recovering them exactly. A profiling-based model then maps reconstructed traces to (a) query attributes from prefill traces (Query Attribute Inference) and (b) generated tokens from decoding traces (Autoregressive Token Recovery), without the attacker ever accessing the victim's actual KV contents, model, prompts, or outputs.

Preconditions / Affected

Requires a shared/multi-tenant GPU where an attacker process can co-reside with the victim's LLM inference job and observe L2 cache / TLB contention. Affects LLM serving stacks using sparse attention mechanisms (tested against three architectures, three sparse-attention schemes, and three sparsity-selection strategies: activation-based saliency, block-level scoring, and learned predictors).

Caveats

The page reports SpaRLeaK "retains over 90% of its original attack effectiveness" under realistic system noise and model evolution, but gives no numeric effectiveness floor for the mitigation tested — it states only that injecting input-independent randomness into sparsity patterns reduces attack success at the cost of degraded inference quality, with no specific tradeoff numbers given in the fetched text. Table 1 in the paper also explicitly marks this attack as having no reported prevention or detection mechanism to date, unlike several prior side channels it compares against.

Organisations named here

  • Sparsity-Induced Memory Access

Technologies named here

  • SparLeak
  • LLM

Techniques named here

  • Sparsity-Induced Memory Access side channel
  • Query attribute inference from prefill-phase traces

Related in AI Security