Verifiable Retrieval for Legal Reasoning through Structure-driven Knowledge Graph

What it claims

A structure-parsed knowledge graph of German federal statutes lets you chunk law text at fine structural units. Chunking at the sentence (Satz) or numbered-item (Nummer) level recovers about three times more ground-truth court citations per 1,000 words of context than naive 100-word chunks. Chunking at the coarser section (§) level does worse than the naive baseline. One-hop graph expansion helps only marginally. All results are for retrieval only; no answers are generated or evaluated.

How it works

• Graph. A custom regex parser, extended from the authors' earlier work, parses the official consolidated text from gesetze-im-internet.de.
• It covers 58 codes, chosen because they are the codes cited in the OpenLegalData corpus.
• The graph has 168,469 nodes and 240,507 edges over an eight-level hierarchy: Root, Code, Titel, §, Absatz, Satz, Nummer, Buchstabe.
• There are only two edge types: contains_* for the hierarchy and cites_* for in-statute references.
• All 22,253 citation edges start at a Satz and stay within a single code, so the graph is a forest of 58 disjoint code trees under an empty root.
• Each parent node's text includes the text of all its children.
• Retrieval. Statute chunks and queries are embedded with KaLM-Embedding-Gemma3-12B-2511, a 12B-parameter model.
• The query is the case-facts section (Tatbestand) of each court decision.
• The top-k=5 chunks are retrieved, optionally expanded by one hop, and decoded to citations through their node IDs.
• Metric (ToER). ToER is prefix recall against the case's cited provisions, multiplied by 1,000 and divided by the corpus-wide mean chunk length at that granularity.
• A match is credited if all levels specified in both citations agree. Deeper levels missing from either citation count as implicit hits.
• Results are stratified by evaluation depth, from Code down to Buchstabe.
• Ground truth is restricted to codes that exist in the graph.

flowchart LR
  A[Statute text] --> B{Chunking}
  B -->|naive 100 words| E[Embed + top-5]
  B -->|KG node level| E
  Q[Tatbestand query] --> E
  E --> X{1-hop expansion?}
  X -->|none| M[Decode citations, score ToER]
  X -->|parent / cited / hybrid| M

Evidence

The evaluation uses 12,037 held-out BGH cases from OpenLegalData. Scores are ToER at k=5, micro-averaged. The graph covers 90.3% of the gold citations.

• Granularity drives the gain. The improvement comes from chunk size, not from structure-awareness alone: §-level chunking falls below the naive baseline.
• Precision is costly. ToER drops sharply between Code and § depth for every chunking level. For Nummer it falls from 14.23 to 3.25.
• Expansion, without counting its tokens. These figures are at Absatz depth, where gold citations most often end, and do not charge the expanded context to the token budget:
• Citation-edge expansion gives a small, uniform gain: Satz 2.28→2.40, Nummer 2.27→2.40, Buchstabe 2.05→2.17.
• Parent expansion changes nothing except at Absatz chunking, where 1.36 rises to 2.13 because the parent is the whole §.

How to use it

The page does not mention a code or graph release. The inputs it names are both public:

• The OpenLegalData dataset (huggingface.co/datasets/harshildarji/openlegaldata).
• The statute text at gesetze-im-internet.de.

Reproducing the work means re-implementing the parser and running a 12B-parameter embedding model over about 168k nodes.

Limits

• Retrieval only. It does not measure whether the generated answers are faithful or cite their sources correctly.
• The metric is lenient and approximate.
• The prefix rule credits unspecified deeper levels as hits.
• ToER divides by the mean chunk length per granularity, not by the tokens actually retrieved.
• The authors argue that raw recall is monotonic in chunk size, but the captured text does not report raw recall.
• Narrow scope.
• Federal statute only; case law is used only as gold.
• No cross-code edges.
• No temporal versioning (amendments or superseded provisions).
• 9.7% of gold citations are out of scope.
• Truncated page. The fetched text cuts off during the expansion results. The token-charged comparison (Fig. 3b) and the hybrid-strategy numbers are not available here, so the true cost of expansion is unverified.
• Venue. This is a workshop paper (RAGE-KG 2026 at ISWC) with a single embedding model and a single k.