Physical Side-Channel Attacks on Edge LLMs: Extracting Output Tokens and User Prompts

What it is

An EM side-channel attack on on-device LLM inference by Alhalawani, Acar, Liu and Batina (Radboud / University of Amsterdam). The target is LLaMA-3.2-1B-Instruct (Q4_K_M, run via llama.cpp) on a Raspberry Pi 5 with a Cortex-A76 CPU. The leak the paper isolates is the LM Head projection, z_t = W·h_t. It is a large, deterministic, quantized matrix-vector multiply (Q6_K weights, Q8_K activations, ARM NEON dot-product) executed at every decode step. Its emanations correlate with the final hidden state h_t (2048-d), which feeds a 128,256-token vocabulary.

How it works

The attacker is passive and profiling-based. They first collect labeled EM traces on a reference device, each labeled with the output token and its hidden state. Against the victim they place a near-field probe over the CPU package.

flowchart LR
    T[EM trace] --> S[STFT spectrogram 256x256]
    S --> V[Pretrained ResNet-18 / CLIP-RN50, fine-tuned]
    V --> C[Closed-set softmax over 50 words]
    V --> H[Regress hidden state 2048-d]
    H --> L[Multiply by public vocab matrix W]
    L --> R[Token ranking over 128k vocab]
    H --> I[Inversion model]
    I --> P[Semantic prompt reconstruction]

• Profiling attack: the classifier identifies 50 medical/privacy keywords with 97.2% accuracy.
• Hidden-state reconstruction: the regressed ĥ_t is multiplied by the public W. This reaches 94.3% accuracy on profiled classes. For unseen words, the correct token is in the top 100 in 67% of cases, with a median rank of 54.
• Inversion: the recovered hidden state is fed to an inversion model to recover prompt semantics. The prompt content is possible because h_t depends on the preceding context.

Preconditions / Affected

• Physical proximity with a near-field EM probe and oscilloscope, plus a reference device of the same type for profiling.
• No access to the victim's software, OS, memory or network, and no modification of the device.
• The paper names Jetson Orin Nano and Hailo accelerators as edge platforms but tests only the Pi 5. Nothing is shown for them.

Caveats

• The setup is highly controlled. Each reply is exactly two tokens (a fixed lead-in word, then the keyword), so the keyword is always the second generated token. That fixes the timing position of the computation. Free-running generation with unknown token positions is not shown in the text I have.
• The 94.3% figure is for profiled classes. Open-vocabulary recovery is a ranking (67% in top 100), not exact recovery.
• Results cover one model, one quantization and one board.
• The fetched text cuts off during dataset construction. The evaluation details, the inversion results and any countermeasures are not covered here, and I could not confirm whether the paper proposes mitigations.

Takeaway for the decision: this matters if your threat model includes physical access to edge devices running LLMs over sensitive text, such as medical or legal use in public spaces. It is not a remote threat. The paper's argument is that "local inference" should not be treated as a confidentiality guarantee against a physical adversary.