Glance: The Generator Is Its Own Reranker

What it claims

Glance reranks RAG candidates with the generator's own frozen backbone. A rank-16 LoRA plus a 256-dimensional pointer head (about 0.1GiB per skill) scores all candidates from one shared prefill, then the same weights answer. The fetched text adds three things the summary omits. Trained adapters keep improving as candidates rise to 200 (43.4k tokens), reaching 0.717 nDCG@10 against 0.659 for bge-reranker-v2-m3. A dedicated cross-encoder stays much faster per query. Memory currently grows with k times context length.

How it works

Every candidate passage goes into one shared state: "[Passage 1] p1 ... [Passage k] pk". Each passage then gets its own short probe branch of about 30 tokens: "Does Passage i contain information that answers or supports the query: q?". Branches start from the end of the state and do not attend to each other. Because the state holds every passage, each probe can compare its passage with the others. The pointer head compares the hidden state at <decide> with each option's </opt> and returns a softmax over {false, true}. No text is generated.

The backbone is MiMo-V2.6-Distill-Qwen-9B. It mixes Gated DeltaNet linear-attention layers with full-attention layers at 3:1. Linear-attention layers carry one recurrent state, so each branch starts from a copy of every layer's cache, using the Kev runtime. Per-passage caches cannot be stitched in arbitrary order, so the two variants answer differently:

flowchart TD
  A[Query + k passages] --> B{Adapter scope}
  B -->|all tokens: main| C[Shared prefill with adapter]
  B -->|probe tokens only: Activated| D[Shared prefill, base weights]
  C --> E[Probes + pointer head score k]
  D --> E
  E -->|main| F[Re-prefill top-n, adapter off]
  E -->|Activated| G[Continue cache, all k kept]
  F --> H[Answer]
  G --> H

The main variant re-prefills about 1.5k tokens. The Activated variant keeps the losing passages in its cache. All results except §4.7 use the main variant.

Evidence

Training starts from a general decision adapter, trained for 20 minutes on 4.4k non-retrieval records. It then runs on 4,000 MS MARCO queries, each paired with one positive and 7 to 19 Tevatron BM25 hard negatives. That is 500 steps, 55,598 probes and 7.7M tokens, with lr 5e-5 (1e-4 for the head) and batch 8. It took 39 minutes on one B200 at 21GiB peak. No BEIR or HotpotQA data was used.

On BM25 top-20 with the zero-shot adapter, packed_all scores 0.715 on SciFact and 0.341 on NFCorpus. It takes 614 and 670ms per query on a shared RTX PRO 6000. The same 9B model making 20 separate calls scores 0.729 and 0.323 at 2210 and 2485ms. bge-reranker-v2-m3 scores 0.721 and 0.326 at 31 and 60ms. After training, packed_all takes 420ms (SciFact) and 411ms (NFCorpus) on a B200. A further 14 minutes on 20 to 80 passage records ("trained-long") gives 0.749 and 0.346.

Scaling used the first 100 SciFact queries with 100-word passages:

The zero-shot adapter peaks at k=20 and falls to 0.584 at k=100.

The HotpotQA test used 300 distractor questions and top-3 context. With the adapter off, Glance context gives F1 0.718 against 0.683 for the cross-encoder, a gap the authors call not significant. All 10 paragraphs give 0.716, and the cross-encoder's recall@2 is higher (0.863 vs 0.762). With the adapter left on, F1 reaches 0.757 and answers are shorter. Against cross-encoder context under the same adapter-on generator, the gap is +0.055 F1 (95% CI [0.018, 0.093]).

How to use it

Adapters and code are at https://huggingface.co/airun-ai/GlanceRAG-MiMo-9B-Reranker. Glance builds on the Kev runtime's shared-prefix path. The adapter stays unmerged and is switched per request. The page states no install commands or licence.

Limits

Evaluation covers two BEIR datasets (about 300 queries each), one QA set and one backbone family. BEIR gaps are not significance-tested, and the authors treat gaps under about 0.02 as indicative. Zero-shot latencies come from a shared GPU, so the ratios between methods are approximate.

Batching across queries lowers throughput, from 2.87 queries/s at B=1 to 1.57 at B=16, and runs out of memory at B=32. A single call runs out of memory at k=200 under a 64GiB cap. A block-mask fix for this is left to future work.

The abstract's +0.045 F1 (CI [0.014, 0.076]) does not match the +0.055 adapter-on gap or the 0.035 adapter-off gap in §4.5. The fetched text is cut off before the Activated-variant results in §4.7.