A Matryoshka Hierarchical RAG for Efficient Multi-Hop Question Answering

What it claims

MatRAG builds a multi-hop RAG index with no LLM calls and no knowledge graph. Internal hierarchy nodes are plain centroids, stored at shorter Matryoshka prefixes as the clusters get coarser. The authors report it achieved the highest Exact Match and F1 on HotpotQA, 2WikiMultiHopQA and MuSiQue against seven RAG baselines. They also report consistently better Recall@5 than the document-retrieval baselines, lower average response time than flat FAISS retrieval, and "substantially" lower indexing cost than graph-based and hierarchical approaches. The fetched text gives no actual numbers.

How it works

Offline indexing

• Each document is encoded once with a Matryoshka (MRL) embedding model and stored at full dimension M. Lower-dimensional views are taken as prefixes on the fly.
• Named entities are extracted per document with GLiNER, an open-schema NER model, and stored alongside the embeddings.
• The hierarchy is built bottom-up with HDBSCAN. Level L holds the documents and level 1 is the coarsest.
• Each node links to its p nearest clusters in the level above, by cosine similarity to the centroids. The result is a DAG rather than a tree. This redundancy makes the descent tolerant of a poor centroid choice at any one level.
• A level-l centroid is the mean of its children, truncated to dimension m_l. Unlike RAPTOR, no LLM summary is generated for internal nodes.

Online retrieval

flowchart TD
  Q[Query + its entities] --> T[Top-down DAG descent<br/>query truncated per level]
  T --> R[Leaf re-rank:<br/>similarity + entity overlap]
  R --> S[Add top k_t docs]
  S --> F{New frontier entities?<br/>budget K left?}
  F -- yes --> C[Cumulative query =<br/>query + retrieved text]
  C --> T
  F -- no --> A[Return context]

• The descent narrows to the most promising cluster at each level until it reaches the documents.
• The traversal score blends similarity to the original query with similarity to the cumulative query. An anchoring schedule increases the weight on the original query over iterations to limit query drift. This follows the classic pseudo-relevance-feedback mitigation of mixing the original and expanded queries.
• Each iteration retrieves as many documents as there are new "frontier" entities from the previous step: entities seen in the retrieved documents but not in the query. The total is capped by a user-set budget K.
• Entity counts replace the LLM planning calls used by plan-then-retrieve methods.

Evidence

• Benchmarks: HotpotQA, 2WikiMultiHopQA and MuSiQue.
• Metrics: EM, F1, Recall@5, average response time and indexing cost.
• Baselines: the fetched text does not list which seven were used. Related work discusses GraphRAG, LightRAG, HippoRAG2, KGP, RAPTOR, ToG and others, but the page never confirms which of these were evaluated.

Limits

• Truncated page. The fetched text ends mid-Section 3. The results tables and the authors' limitations section (Section 5) are missing, so every claimed gain above is unquantified here.
• Unstated settings. The page does not name the Matryoshka embedding model, or the values of L, m_l, p, K or the anchoring weights.
• No release. There is no mention of code, a repository or a licence.
• Entity dependence. The retrieval budget is driven entirely by NER output. GLiNER misses or spurious entities directly change how many documents are fetched, so check this on corpora that are not entity-centric.
• Latency comparison. The speed claim is stated against flat FAISS. A latency comparison against the graph-based and hierarchical baselines is not stated in the fetched text.