Skip to content

Back to the catalog

Investigating the Role of Reasoning-Language Alignment in Monolingual Retrieval-Augmented Generation
arxiv.orgpaper

Investigating the Role of Reasoning-Language Alignment in Monolingual Retrieval-Augmented Generation

This paper studies whether forcing an LLM to reason in a specific language helps or hurts accuracy in retrieval-augmented generation (RAG), extending prior findings from short-prompt reasoning to settings where the model must integrate large retrieved context. The authors build a fully monolingual German RAG QA benchmark set in the fictional world of the tabletop RPG "The Dark Eye" — a domain richly documented in German but too niche for the model to answer from memory — forcing retrieval rather than memorization. They find that aligning reasoning language with the query/document language (German) outperforms a mismatched but higher-proficiency language (French), showing the benefit comes from language alignment rather than raw language skill, though forced German reasoning still only matches, and never exceeds, the model's native unconstrained English reasoning. The testbed and QA benchmark are publicly released.

TL;DR Forcing an LLM's agentic RAG reasoning to match the query/document language (German) beats a higher-proficiency mismatched language (French), but still can't beat native English reasoning — and the team releases a German "Dark Eye" RAG benchmark to prove it.
Open original ↗
Document

What it claims

Using Qwen3-30B-A3B in an agentic RAG loop over German sourcebooks of The Dark Eye, forcing the model to reason in German beats forcing it to reason in French, even though the model benchmarks higher in French. The authors read this as an effect of alignment with the evidence, not of language proficiency. Two findings go beyond the summary:

  • Chunking changes the effect. The alignment benefit grows when chunks follow section structure rather than fixed sizes. Section-based chunks help forced German the most, and they shrink the general accuracy cost of forcing any reasoning language.
  • Reasoning only pays off on hard questions. With reasoning disabled, the system does best on simple single-passage questions and worst on complex multi-hop ones.

Without retrieval, the model scores near the minimum, so the questions do require retrieval.

How it works

The agent loop is standard and built without an agent framework. On each turn the LLM sees the system prompt, the query and the evidence gathered so far. It then either calls a tool or answers, and the answer is always in German.

flowchart TD
  Q[German query] --> L[LLM reasons in forced language]
  L -->|tool call| S[RAG search: top-30 → rerank → top-3]
  L -->|tool call| D[Fetch chunk by ID: parent/child]
  S --> L
  D --> L
  L -->|done| A[Answer in German]
  • Retrieval: Multilingual-E5-large embeddings with cosine similarity keep the top 30 chunks. Qwen3-Reranker-0.6B then reranks them to the top 3. A per-query blacklist stops already-seen chunks from coming back.
  • Navigation tool: a separate direct-retrieval tool fetches a chunk by ID, so the model can walk to a parent or child section.
  • Section chunks: each chunk is one section, carrying its ancestor headings and links to its parent and child chunks.
  • Language forcing: this follows Qi et al. (2025). A system-prompt instruction is combined with a fixed prefix in the target language, prepended to the reasoning trace (e.g. "Ich muss zunächst durchgehen, was ich über die Anfrage des Nutzers bereits weiß.").
  • Conditions compared: forced German, forced English, forced French, unconstrained (English) and no reasoning.
  • Chunking baseline: SimpleChunks uses fixed 501-token chunks, sized to match the mean of the section chunks. It has 2,250 chunks against 2,246 for SectionChunks.

Evidence

  • Corpus: three "green series" region books (G01, G05, G08), about 1.13M tokens in total. That is far beyond the model's context window.
  • Single-passage set: 585 questions. An LLM generator proposed them, a critic model rated them, low-rated pairs were revised, and a human reviewed each one.
  • Multi-hop set: 30 hand-written questions that draw on all three books, judged by human experts.
  • Judge model: Qwen3-235B-A22B-Thinking, which also serves as the no-retrieval baseline.
  • Decoding: temperature 0.6, top_p 0.95, top_k 20, min_p 0, fixed seed.
  • Run coverage: SimpleChunks was run only on the single-passage set with forced German, forced English and unconstrained reasoning. SectionChunks was run with all five settings on both sets.

How to use it

  • Code and benchmark: https://github.com/soliver1/reasoning-language-alignment, containing the QA benchmark, the system code and the PDF-to-knowledge-base tool.
  • Hardware: the main RAG runs use a single 24 GB GPU.
  • PDF extraction: Docling, Marker and pdfalto all failed on the books' two- and three-column layout. The authors' pipeline uses pdfalto output plus manual correction of heading levels and block order, so expect manual work if you adapt it to other documents.

Limits

  • No redistributable knowledge base. The sourcebooks are copyrighted. You need to own the original PDFs to rebuild the corpus.
  • Narrow setup. The study covers one RAG model, one target language and three books. The multi-hop set has only 30 questions.
  • No native multilingual reasoning. Prompt-based forcing narrows the gap to English but does not close it. The authors point to native multilingual reasoning as the fix and do not test it.
  • No result numbers. The fetched text stops at Table 1, the knowledge-base statistics, so no accuracy figures or metric definitions are available here. Check the full paper before relying on effect sizes.

Organisations named here

  • Mario Sanz-Guerrero
  • The Dark Eye

Techniques named here

  • Agentic RAG with forced reasoning language
  • Monolingual RAG testbed construction

Related in research