Hybrid Reasoning Agents

What it claims

HybridAgent is a modified autoregressive LLM that switches between ordinary token generation and latent reasoning, where hidden states are fed back as inputs without decoding them. Tool calls stay in text. The authors build it from Qwen3-Base at 4B, 8B and 14B. Against text-based and latent agent baselines across 11 benchmarks, they report an average +10.4% improvement, 35.5–54.7% fewer tokens, 1.8–2.6× faster inference and 2.7–3.7× lower cost. They also report that latent spans can be read back as "pseudo-text". Code is at https://github.com/togethercomputer/HybridAgent.

How it works

HybridLink. This is a residual MLP adapter, e_{t+1} = h_t + W2·GELU(W1·h_t). It maps the last-layer hidden state back into the input-embedding space. Earlier latent reasoners feed h_t in raw, which the authors argue mismatches the pretrained input space. The adapter has about 2·d_h² parameters, roughly 10M for a 4B model. The backbone architecture is otherwise unchanged, so no pre-training is needed.

Mode switching.

• The model enters latent mode by emitting a special <latent> token.
• In latent mode, each step's output is passed through HybridLink and used as the next input.
• The model exits when the HybridLink output's nearest token embedding (by cosine similarity) is </latent>.
• The same nearest-token lookup turns any latent step into readable pseudo-text. This is the "inspectable" claim.
• Span lengths and switch frequency are decided by the model, not preset.

flowchart LR
  T[Text mode: sample token] -->|emits <latent>| L[Latent mode: h -> HybridLink -> next input]
  L -->|nearest emb != </latent>| L
  L -->|nearest emb == </latent>| T
  T -->|tool call text| Tool[Tool execution]
  Tool -->|observation appended to KV| T

Working memory. A hybrid trajectory cannot be re-tokenized, because latent vectors have no tokens. So multi-turn context is carried as the accumulated KV cache across text, latent and tool-observation positions, and generation resumes from it each turn. The authors give a formal argument that this is lossless (Appendix A).

HybridSFT.

• Data: existing text agentic SFT datasets are filtered and converted to hybrid traces. Steps are assigned to latent mode using LLM-expert judgments and the target model's own prediction entropy. A small set of key steps stays in text for tool use and auditing.
• Activation: plain text SFT first teaches the backbone to emit the anchor tokens. The backbone is then frozen and HybridLink is trained with a weighted cosine loss plus an MSE loss against the ground-truth token embeddings at latent positions.
• Self-conditioned training: the first pass is teacher-forced and produces HybridLink predictions. A second pass substitutes those predictions for the ground-truth latent embeddings. Backbone and HybridLink are then trained jointly, with cross-entropy at text positions and regression at latent positions. This targets the mismatch between training and inference.

HybridRL. This stage learns when to think latently and when to act in text. The page names "hybrid policy factorization" and "exact rollout replay" over latent context but does not detail them in the fetched text.

How to use it

The page gives the repository link and the backbone family (Qwen3-Base 4B/8B/14B). It gives no install commands, released weight names or licence. Two serving constraints follow from the design:

• Your inference stack must allow hidden-state feedback through an adapter, which standard decode loops do not.
• Context must persist as KV cache between turns rather than as a text transcript.

Limits

• The fetched text cuts off partway through Section 4.1. HybridRL details, the 11 benchmark names, per-benchmark results, baseline identities and ablations are not visible. The headline numbers therefore cannot be tied to specific tasks or baselines here.
• "Interpretable" means nearest-token projection of latent steps. Readability is shown in Figure 2 and Appendix F, which are not in the fetched text.
• Because KV cache is the working memory, a conversation cannot be rebuilt from a stored text log, and context cannot be truncated or edited the usual way.
• No licence is stated for the code or for any weights.