AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines

What it claims

AgSpec is a training-free policy layer that sits on top of an existing retrieval-based speculative decoding engine, such as SAM-Decoding's suffix automaton or SuffixDecoding's suffix tree, and does not modify that engine. The authors argue that existing drafters fail in coding agents for two reasons:

• Availability. Reusable text has dropped out of the corpus: the prompt truncated a file, or an earlier call has ended.
• Matchability. The corpus stores text as the agent read it, but the agent emits it in another form, for example diff-prefixed lines.

The headline case is Gemma3-27B on SWE-bench Verified at batch size 16. Plain SAM-Decoding reaches 2.72× over autoregressive decoding; AgSpec-SAM reaches 3.61×.

How it works

Three corpora, kept separate by provenance and lifetime:

• Session corpus. Prompts, tool outputs and generated tokens, shared across all calls and agents in the session, so retries can copy earlier attempts.
• Workspace corpus. Repository files opened during the session. It grows as files are opened and is discarded when the session ends.
• Global corpus. A static reference body that is built before serving and stays fixed.

Emission-form indexing. When a file is first opened, it is indexed in several forms:

• As is, so it stays retrievable after prompt truncation.
• With every line prefixed by a space, matching kept lines in a unified diff.
• With every line prefixed by a minus, matching removed lines.

Scaffolds that write whole files through a tool call get one more copy, with newlines and quotes escaped. The output format is a property of the scaffold, so it is known before the session runs.

Source preference. The engine reports the longest match in each corpus: mS (session), mW (workspace) and mG (global).

flowchart TD
  Q[Context suffix] --> M[Longest match per corpus: mS, mW, mG]
  M --> G{mG > max of mS, mW + δ?}
  G -- yes --> DG[Draft from global]
  G -- no --> S{mS ≥ mW?}
  S -- yes --> DS[Draft from session]
  S -- no --> DW[Draft from workspace]

The margin δ exists because a long global match is likely coincidental. A δ sweep is in appendix C, which is not in the fetched text.

Continuation selection. A match can occur in several places with different continuations.

• In the session and global corpora, occurrences are grouped by their next 3 tokens; the most frequent group wins, and ties go to the most recent occurrence.
• In the workspace corpus, the first occurrence is used.

Draft length. Each agent role (localizer, coder, executor, reviewer) gets a per-agent draft cap, profiled offline. A learned scale then adjusts the length per turn and per request from verification feedback. The motivation is that accept length varies widely by role and drifts across turns, and rejected drafts become costly at large batch sizes where verification is compute-bound.

Integration requirement. Any engine works if it can search multiple corpora and accept a per-step draft length.

Evidence

• Accept-length study (Fig. 1a). This uses Devstral, Gemma3 and Qwen3.6 on SWE-bench Verified. Widening the retrieval scope from prompt to artifact to session to repository history plus emission forms raises accept length on every model. Session history gives the largest gain. Repository history and emission-form indexing add smaller headroom.
• Ablation (Fig. 1c). Gemma3-27B, SWE-bench Verified, batch size 16, speedup over no speculation:

The fetched text does not say whether the two partial rows are each applied alone to SAM-Decoding. Either way, corpus changes contribute more than length control.

• Baselines. AgSpec is compared against autoregressive decoding, PLD, REST, SAM-Decoding, SuffixDecoding, FastCoder and EAGLE-3 on SWE-bench and TeamBench. It wins "in most evaluated settings."

Limits

• The fetched text stops partway through §3.2. The exact draft-length controller is not shown: how the scale is learned, its update rule, and how the offline cap is profiled. Also missing are the full result tables, the per-model numbers behind the 4.37× and 4.76× peaks, and the settings where AgSpec loses to a baseline.
• Using AgSpec requires an offline profiling pass per agent role, plus knowledge of the scaffold's emission format. Formats other than unified diff and escaped whole-file writes are only named, such as search-and-replace blocks and line-addressed edits. The fetched text does not say they are indexed.
• The paper is a v1 preprint dated 1 Oct 2026. No code, licence or install instructions appear in the fetched text.