🎯 Speculative decoding

The items that name it 25

• How to Run an AI Model Locally—And Why You Might Want To
  article2026-11-01

  A small model proposes several tokens ahead and the large model checks them together. Google's Multi-Token Prediction drafters for Gemma 4 use it to speed up generation by up to 3x.

• [AINews] Building AIE CODE: The top Coding Agent Conference
  article2026-10-30

  vLLM's built-in DSpark drafter gives 1.92 to 2.23x output throughput for Mistral Large 4.

• TextSeal2: Distortion-Free LLM Watermarking with Copula Mixing and Memoryless Races
  paper2026-10-29

  Drafts tokens that the target model verifies. TextSeal2 keeps tokens that replace rejected drafts watermarked with the same keys, without changing the acceptance rate.

• Only Accepted Tokens Count: An Energy Accounting of Speculative Decoding Under Latency Targets
  paper2026-10-29

  A drafter proposes several future tokens and the target model verifies them in a single forward pass that reuses the same weight read, making an individual user faster.

• Multi-Token Prediction Done Right
  paper2026-10-28

  Faster inference that MTP heads enable; the KL self-distillation term improves it.

• Evaluating the Applicability of LLM Inference Optimizations on Apple Silicon
  paper2026-10-22
• RIFT: KV Cache Compression for Draft Models with Complementary Compressibility Training
  paper2026-10-22
• The Huawei-DeepSeek Symbiosis
  article2026-10-21

  Lossless method that predicts draft tokens and verifies them in a single pass, amortizing weight loading. DSpark drafting is much faster than the verification step.

• [AINews] High Taste Testers Ep. 1
  article2026-10-17

  DFlash2 drafter with Reflection Beam and LiLiCorr/DFlash with Nemotron Bolt to raise throughput.

• [AINews] High Taste Testers Ep. 1
  article2026-10-17
• Acceptance Is Not Speed: Length Inflation in Training-Free Lossy Speculative Decoding
  paper2026-10-16
• DLite: History-free Speculative Decoding for Agentic AI
  paper2026-10-16
• SP$^2$ec: Adaptive Self-Speculative Decoding for Vision-Language Models
  paper2026-10-13It is written here as Speculative Decoding (SD).

  SD

• Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds
  paper2026-10-07
• [AINews] Opus 5.5 is good at explainer videos
  article2026-09-29It is written here as Speculative decoding (DSpark).

  DSpark

• [AINews] The Future of Latent Space
  article2026-09-25It is written here as Speculative decoding (DSpark).

  DSpark

• The Roadmap to Mastering LLM Inference Optimization
  article2026-09-21
• Nehanth/pooled
  repo2026-09-01
• peonist-ai/halogen-flash-server
  repo2026-08-26

  Used for speed with two draft sources, the model's own draft head and the request's own text (prompt lookup). At temperature 0 output is byte-identical to serial greedy decode, verified on every release.

• Up to 3.2x Faster Inference with LFM2.5-DSpark
  article2026-08-20
• patchy631/time-to-first-token
  repo2026-08-02
• deepbeepmeep/Wan2GP
  repo2026-07-30
• Tencent Hy4 preview (open-weight MoE + Sherry quantization)
  repo
• vLLM v0.30.0
  repoIt is written here as Speculative decoding (MTP/EAGLE3/DFlash/DSpark).

  MTP/EAGLE3/DFlash/DSpark

• How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Method
  paperIt is written here as Speculative decoding (inverted).

  inverted