Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

What it claims

Standard pre-trained LLMs, when fed the token-wise average of two documents' embeddings, keep both streams' ground-truth next tokens near the top of the mixed output distribution — without any architectural change. The paper calls this the Superposition Linearity Hypothesis and, critically, shows the effect is strongest at initialization and decays over pre-training, arguing superposition is an architectural bias rather than a learned skill. It is tested on Pythia, Qwen2.5, Llama-3, OLMo and Gemma families, using TinyStories and FineWeb text pairs.

Evidence

On unmodified models the ground-truth token appears in the top-10 in ~30–40% of cases, top-50 in 50–60%, top-100 in 60–65% (vocab ≥50,000). A normalized Superposition Approximation Ratio R_D (mixed-vs-target divergence over unrelated-context divergence) stays <1 across KL, JS and Wasserstein for all tested models (Table 1: e.g. Pythia-2.8B R_KL≈0.42 at L=32). Hidden-state linearity error grows monotonically with training step across all six Pythia sizes (Fig. 2). The effect persists at N=3 streams with only mild degradation (R_KL +0.04 to +0.09).

The caveat the headline hides

The aggregate rank numbers are carried by "predictable" positions (~65%), where the LM-head frequency prior dominates. On the harder content positions (~35%), base-model embedding mixing pushes the true token to median rank 284. The paper's attention-patching controls make this sharp: donor patching (substituting an unrelated text's attention weights) looks benign as self-agreement (median rank 8) but is catastrophic on task — LAMBADA accuracy drops from 73% to 0.5%, target at median rank 2,350. Permutation patching (destroying attention shape) collapses everything. So what survives is the joint effect of the frequency prior plus preserved attention structure — not evidence that two full contexts are cleanly processed in parallel.

What fine-tuning actually buys

The lightweight fine-tuning (the summary's "restoration") uses <0.025% of the pre-training dataset size and its real value is on content tokens: after tuning, Qwen2.5-3B content-position median rank goes 284 → 5 and top-1 agreement 4.2% → 22.8% — i.e. it repairs exactly the hard case the base model fails.

Limits

This is a v1 preprint (arXiv:2609.29845). The disentangling decoder is described but the fetched text cuts off before its results, so the quality of the two recovered continuations is not shown here. Mixing is element-wise averaging of equal-length, truncated sequences; behavior on unequal lengths or non-uniform weighting is not reported. Read the "two coherent continuations from one forward pass" claim against the content-vs-predictable split above before assuming free parallel inference.