Hybrid Reasoning Agents: LLM agents that think in latent space, act through text

What it claims

HybridAgent converts a pretrained LLM into an agent that alternates between continuous latent reasoning and ordinary text within one autoregressive generation. Text is used for tool calls and readable checkpoints. The authors (Stanford, Together AI, UIUC, Meta; arXiv v1, 13 October 2026) build it on Qwen3-Base at 4B, 8B and 14B. They test it on 11 benchmarks covering mathematics, science, medicine, knowledge-intensive search and agentic reasoning. Against text-based and latent agents and frameworks, they report a 10.4% average performance gain, 35.5% to 54.7% fewer tokens, 1.8x to 2.6x faster inference and 2.7x to 3.7x lower cost. The cost figure does not appear in the catalog summary.

How it works

HybridLink is a residual adapter. It maps the last-layer hidden state back into the input embedding space as e = h + W2·GELU(W1·h), which gives about 2·d² parameters (about 10M for a 4B model). The authors use it because feeding raw hidden states back as inputs, as earlier latent reasoners do, creates a mismatch with the pretrained embedding space.

Mode switching is decided by the model, not fixed in advance:

flowchart LR
  T[Text mode: sample token] -->|emits latent token| L[Latent step: h through HybridLink]
  T -->|tool call| E[Tool executes, response appended]
  E --> T
  L --> C{Nearest token embedding is the exit anchor?}
  C -->|no| L
  C -->|yes| T

The model enters latent mode when it emits a special <latent> token. It leaves latent mode when the token embedding with the highest cosine similarity to R(h) is </latent>. The same nearest-embedding lookup turns any latent step into readable pseudo-text, and that lookup is the basis of the inspectability claim.

Hybrid trajectories cannot be re-tokenized, so context does not carry across turns as text. It carries as the accumulated KV cache, which the paper calls "continuous working memory". The model resumes each turn from that cache and appends text, latent and tool-observation entries to it. Appendix A gives a proof that this preserves the full context losslessly.

Training has three stages:

1. Data. Existing text agentic SFT datasets are filtered and converted to hybrid form. Steps go to latent mode based on LLM-expert judgments and the target model's prediction entropy. A few key steps stay in text for tool use and auditing.
2. Activation. Standard text SFT teaches the anchor tokens. Then the backbone is frozen and HybridLink is trained with a weighted cosine plus MSE loss against the embeddings of the ground-truth tokens at latent positions.
3. Self-conditioned HybridSFT. A teacher-forced pass produces R(h). A second pass substitutes these predictions for the ground-truth latent embeddings and jointly trains backbone and HybridLink, using CE at text positions and regression at latent positions. This targets the mismatch between training and inference.

HybridRL follows. It uses "hybrid policy factorization" and "exact rollout replay" of latent context so that policy updates cover complete hybrid trajectories.

How to use it

Code is at https://github.com/togethercomputer/HybridAgent. The method needs no pretraining, only the SFT and RL post-training applied to an existing base model. Inference requires a serving stack that keeps and reuses the KV cache across tool turns, because the context cannot be rebuilt from text. The page states no licence, released weights, install command or hardware requirements.

Limits

The fetched text stops partway through Section 4.1, so the HybridRL details, per-benchmark tables, named baselines and ablations are not visible. The headline numbers here are therefore averages without the settings needed to check them. Interpretability rests on nearest-token pseudo-text. The visible text describes Figure 2's readability metrics only as "highly readable and context-specific" and leaves the details to Appendix F. Only Qwen3-Base backbones are reported.