Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUs

What it claims

Tandem is a llama.cpp-based runtime that serves hybrid linear-attention GUI agents on Qualcomm Hexagon NPUs. It targets three costs that chat-oriented serving handles poorly: recurrent-state checkpoints placed where agent prompts do not diverge, speculative rollback that pushes prefill off the matrix unit, and NPU power votes that keep an idle, loaded model drawing power. Against llama.cpp's fastest NPU configuration on a Snapdragon 8 Elite, text-step latency falls 30%. On the next Snapdragon generation, which the page does not name, the reduction is 27%. Code is at https://github.com/flyingdoog/tandem.

How it works

The test policy is Qwen3.5-4B, 4-bit, LoRA fine-tuned on public GUI-agent data plus on-phone demonstrations. It has 32 layers: 8 full-attention layers and 24 Gated DeltaNet layers. The recurrent state is 50.25 MiB per sequence regardless of prompt length. Attention adds 32 KiB of KV cache per token.

A recurrent state cannot be truncated the way a KV cache can, so prefix reuse needs a checkpoint at the exact shared position. Stock llama.cpp checkpoints at user-message boundaries and near the prompt end. Agent prompts keep the rules and task but rewrite the last three actions and the screen controls, so they diverge earlier. Tandem's changes:

• Divergence-point checkpoints save state where consecutive prompts split, and input batches are consolidated to avoid extra passes.
• Rollback-aware prefill keeps most prefill on the HMX matrix unit and produces rollback states only through a short per-token tail.
• Column-shared verification reads each weight block once for all draft tokens from the model's MTP head. A trimmed draft vocabulary and fused operators cut proposal cost and memory traffic.
• Between request bursts, Tandem releases the session's NPU power votes but keeps the weights loaded.

Evidence

The workload is an "app set" of 21 screens, one from each of 21 Chinese apps, plus a replay trace that adds 40 steps from five recorded episodes.

A text prompt averages 648 tokens: 324 tokens of rules, 49 of task and history, and 275 of controls. Its answer is about 14 tokens. Screenshot prompts average 1,778 tokens, and they take 4.1× as long as text prompts. The vision encoder spends 1.70 s producing 1,127 image tokens, and prefilling those tokens takes another 1.66 s. This image cost is why screenshot gains (16%) trail text gains (30%).

A calibrated prefill cost model gives 1.30 ms per token plus 86 ms per pass. Ending a batch early therefore costs about as much as processing 66 more tokens. On the replay trace, the stock checkpoint policy reprocesses an average of 111 shared tokens per subsequent request.

Speculation needs Tandem's prefill and verification changes to pay off. Enabling MTP with llama.cpp's existing rollback and verification paths adds 0.67 s per text step, and most of that comes before generation starts.

Energy per text step falls 33%. Standby power with the model loaded falls from 0.76 W to 0.22 W.

How to use it

Tandem is implemented in llama.cpp's server and Hexagon backend. The tested NPU is Hexagon v79 on the Snapdragon 8 Elite, which has 6 HVX threads, an fp16 HMX unit and 8 MB of VTCM. The fetched text gives no build commands or licence.

Limits

The fetched text ends partway through §3, so the end-to-end methodology, results for other model sizes and recurrent architectures, and the action-fidelity results it mentions were not seen. The workload is small: 21 screens and 40 replayed steps. The phase-breakdown profile used fan cooling. The paper is a preprint dated 19 October 2026.