Up to 3.2x Faster Inference with LFM2.5-DSpark

What it claims

Liquid AI trained ~300M-parameter DSpark draft models for three LFM2.5 targets: 2.6B, 1.2B-Instruct and 8B-A1B (MoE). It reports up to 3.18x throughput on an H100 and up to 2.87x on an M4 Max. The speedup is uneven across targets:

• LFM2.5-2.6B gains the most on-device, reaching about 140 tok/s on the MacBook depending on dataset.
• LFM2.5-1.2B-Instruct varies by up to 52% depending on the dataset's text distribution.
• LFM2.5-8B-A1B gets only an 18% average on-device gain, despite having a higher acceptance rate than the two dense models.

Output is identical to plain greedy decoding, so accuracy is unchanged. That guarantee holds only under greedy decoding.

How it works

The draft model has three parts:

• Parallel backbone (DFlash-style). It is conditioned on the target model's context features and produces hidden states for all draft tokens in one forward pass.
• Sequential head. It models a Markov chain between neighbouring tokens, which raises acceptance at later draft positions.
• Confidence-scheduled verifier. It predicts each token's survival probability and prunes low-confidence suffixes when verifying them would cost more than it saves.

The released drafters are simplified attention-only models with 5 layers and a block size of 9. They were trained on a data mix covering SFT, chat, code and function-calling, for 15 epochs. The checkpoint kept was the epoch with the highest acceptance rate, not the lowest loss. When a draft token is rejected, the target model's own token replaces it.

Evidence

All results use batch size 1, temperature 0, block size 9, and five benchmark datasets. The page does not name the datasets, and the text gives no per-dataset numbers.

• GPU: SGLang on one H100 80GB, BF16. Best case 3.18x over the same server without speculative decoding.
• On-device: llama.cpp with experimental Metal kernels on an M4 Max MacBook Pro, FP16 GGUF, up to 256 output tokens. Best case 2.87x.
• Function calling: latency drops 57% on average across multi-tool scenarios, measured for LFM2.5-2.6B only.

How to use it

Both runtimes need a build that includes a specific PR.

SGLang needs PR #31041. The block size is read from the draft's config.json. For a baseline, run the same command without the three --speculative-* flags. The server exposes an OpenAI-compatible endpoint at http://localhost:30000/v1.

python -m sglang.launch_server \
  --model-path LiquidAI/LFM2.5-2.6B \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \
  --speculative-draft-attention-backend flashinfer \
  --disable-radix-cache --mem-fraction-static 0.75 --port 30000

llama.cpp needs PR #27383. The block size is read from sidecar metadata, and n-max is clamped to it. Per-response timings report draft_n and draft_n_accepted, which lets you check acceptance on your own traffic.

llama-server -m LFM2.5-2.6B-F16.gguf \
  -md LFM2.5-2.6B-DSpark-F16.gguf \
  --spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \
  -fa on -ngl 99

The checkpoints are on Hugging Face as LFM2.5-{2.6B,1.2B-Instruct,8B-A1B}-DSpark, with -GGUF variants. The page does not state a licence.

Limits

• MoE on Metal gains little. The authors attribute the 18% result to two causes: llama.cpp's current MoE implementation on the Metal backend, and the fact that verifying k tokens activates more experts, which means more weight traffic than a single decode step.
• Only greedy, single-request serving was measured. All numbers are at temperature 0 and batch size 1. The page says nothing about sampled decoding or batched serving.
• The SGLang command disables the radix cache. The page does not discuss whether that affects prefix-caching workloads.
• The Mac results depend on experimental kernels. The llama.cpp path relies on experimental Metal kernels, and both integrations need the PR builds above.
• Speedup depends on your data. The 1.2B result shows it, so measure acceptance on your own distribution before relying on the headline figures.