Skip to content

Back to the catalog

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
arxiv.orgpaper

Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents

The paper introduces Selection-based Structured Reasoning (SSR), a framework that replaces free-form chain-of-thought generation in multimodal search agents with selection among pre-specified, reusable natural-language reasoning candidates, scored by likelihood via teacher-forced prefilling with a shared KV cache. Evaluated on seven multimodal search benchmarks using 2B and 4B scale models under both reinforcement learning and supervised fine-tuning, SSR matches the success rate of leading search agents of the same scale while cutting per-turn reasoning latency by over 90% and total per-question inference latency by 28-54%. It targets the problem of small models producing long, low-value reasoning traces before each action, which is costly at inference time.

TL;DR A new framework turns agent "reasoning" into picking from pre-written candidate thoughts instead of generating free text, cutting multimodal search-agent latency by up to 90% with no accuracy loss.
Open original ↗
Document

What it claims

Apple and CMU authors replace a ReAct agent's free-form <think> span with a choice from a short library of hand-written natural-language reasoning candidates. Their observation is that in multimodal search the high-level rationale recurs across questions even when the entities differ. For example, "entity known, attribute missing → search entity + attribute" covers many questions. The headline result is that the 4B SSR model trained with GRPO averages 61.37% success across seven benchmarks, against 61.25% for leading trained 4B search agents. The paper also reports that SSR beats four efficient-reasoning baselines on success rate and inference time, with more consistent latency.

How it works

  • Scoring. Each candidate rᵢ is scored by length-normalized log-likelihood under the acting model itself: S = (1/Kᵢ^α) Σ log p(rᵢ,ⱼ | hₜ, rᵢ,<ⱼ). The authors use α = 1, which is mean token log-prob. Without normalization, scoring favours the shortest candidate.
  • Selection. A softmax with temperature τ over the scores gives a categorical policy. One candidate is sampled and inserted verbatim as the reasoning trace. The same model and system prompt then generate the action autoregressively: a search query, a crop location or a final answer. No classifier head or extra component is involved.
  • Parallelism.
  • All candidate tokens are teacher-forced in a single prefill.
  • All candidates share the history KV cache.
  • The authors argue critical-path depth drops from O(L·log(H+L)) to O(log(H+K_max) + log N), which removes the dependence on reasoning length L.
  • Training.
  • SFT uses a weighted sum: λ × the categorical selection loss on the ground-truth candidate, plus a length-normalized action token loss.
  • RL (GRPO, GSPO, SAPO) replaces the per-token importance ratios over generated reasoning with one categorical ratio q_θ/q_old. Action tokens keep their usual ratios.
  • Because reasoning is confined to the library, RL works without an SFT warm start. The authors contrast this with prior agents such as DeepEyesV2.
flowchart LR
  H[History KV cache] --> A[Candidate 1 prefill]
  H --> B[Candidate 2 prefill]
  H --> C[Candidate N prefill]
  A --> S[Softmax over mean log-prob]
  B --> S
  C --> S
  S --> I[Insert chosen text]
  I --> G[Autoregressive action]

Evidence

  • Training objectives. Results cover GRPO, GSPO, SAPO and SFT at 4B, and GRPO at 2B. Across all of these, success is comparable to or higher than the free-form equivalent.
  • Latency. Mean reasoning latency per turn falls by more than 90%. Total model latency per question falls by 28–54%; the paper gives the per-setting breakdown in its Table 2.
  • Worked example.
  • The free-form trace used 156 tokens and took 1821.92 ms. It misidentified the building as the National Museum of Scotland and issued a wrong text search.
  • The SSR trace used 31 tokens and took 32.66 ms. It selected "Identify by image" and called reverse image search, which correctly found Old College, University of Edinburgh.

How to use it

The fetched text gives no code, weights or licence, only a project page: https://zfy0314.github.io/ssr-webpage/. To adopt SSR you need:

  • a candidate library (examples include "Identify by image", "Look up a fact" and "Answer from image");
  • an inference stack that can prefill several continuations against one shared KV cache and return per-token log-probs.

Limits

  • The fetched text is cut off partway through the RL objective. That means the benchmark names, base models, library size, how the library is built, Table 2, ablations and the authors' stated limitations are all unavailable here. Check the full paper for these.
  • Reasoning is hard-constrained to the library. The paper does not show what happens when no candidate fits, or whether the method transfers beyond search tasks with recurring rationales.
  • Only 2B and 4B models are tested.
  • The latency gains apply to the reasoning span only. Action tokens are still decoded autoregressively, which is why per-question savings (28–54%) are much smaller than per-turn reasoning savings (more than 90%).

Technologies named here

Techniques named here

Related in research