RECAST: Learning to Compute the Right Context through Adaptive Evidence Routing

What it claims

Evidence for a RAG answer often has to be computed, not just retrieved. A trained 9B router (Qwen3.5-9B) that chooses between retrieval, SQL and generated Python reaches 75.6% mean success on six benchmark families. The strongest large-model baseline gets 59.7%. The same pipeline with an untrained Gemini 3.5 Flash router gets 70.6%. The summary's "15.9%" and "5.0%" are absolute percentage-point gaps between these figures.

How it works

flowchart TD
    S[Source] --> P[Rule-based preprocessor:<br/>records + source profile]
    P --> R{RouterLM}
    R -->|CALL_PRIMITIVE| PR[BM25 / BGE-M3 / read-only SQL]
    R -->|SYNTHESIZE| C[CompilerLM writes Python] --> X[Execute]
    PR --> F[Evidence + outcome]
    X --> F
    F --> R
    R -->|ACCEPT_CONTEXT| A[AnswerLM]

• Preparation. A deterministic preprocessor uses no LLM. It flattens any source (dataframes, financial reports, hierarchical tables, passages, user profiles) into a uniform list of records. It also builds a compact profile of scale, structure, formats and fields. The router sees only this profile, never the full source.
• Primitives:
• Lexical is BM25 and takes a query plus top_k.
• Semantic is cosine similarity over BGE-M3 embeddings.
• Relational is a read-only SQL query over the normalized records, supporting joins, grouping, aggregation and arithmetic.
• Any call can be restricted to record IDs found in earlier rounds.
• Synthesis. The router writes a spec with these fields:
• evidence_goal
• required_output (including source IDs)
• reason (why the primitives are insufficient)

The CompilerLM turns the spec into one Python program in a single call. It sees the spec, the source profile, source samples and prior outcomes, but not the original task, so the spec must carry every task constraint. The program can process the full source and call the primitives.

• Feedback. Every action returns its execution status, match and return counts, and any errors. Evidence accumulates as a set union, and the action and outcome are appended to the history. Each router action is JSON with an explicit reason field.
• Training. Only the RouterLM is trained, with SFT followed by GRPO. The CompilerLM and AnswerLM stay frozen, and the AnswerLM is called once with the accepted evidence.

Evidence

• In-domain benchmarks: DataBench, FinQA, HiTab, HotpotQA, LaMP, MultiHiertt.
• Held-out benchmarks: 2WikiMultiHopQA, TAT-QA, WikiTableQuestions.

The authors also claim the router transfers effectively across alternative CompilerLMs. The fetched text gives no numbers for that.

Limits

• The fetched text stops in Section 3.3.3. It contains no per-benchmark tables, ablations, baseline identities, training data or compute figures, and no cost or latency figures.
• It does not say which models serve as the CompilerLM and AnswerLM.
• It does not say how the training traces for SFT were produced, or what the GRPO reward is.
• It names no code, weights or licence.
• The synthesis path executes LLM-generated Python over the full source. The text describes no sandboxing.
• The figures are mean success rates. Per-dataset scores, and how success is defined for each benchmark, are not in the fetched portion.