Heading Hierarchy Reconstruction for Complex and Long Documents with Level-Wise Multimodal Memory

What it claims

The method reconstructs heading levels for long documents one heading (or one page group) at a time. At each step the VLM sees at most one historical page per hierarchy level already observed, plus the target. On the authors' new CLDoc-HHRBench it reaches 86.01 Macro / 91.07 Micro STEDS. The best baseline strategy there scores 83.90 / 79.09 (Pseudo-ToC). A fine-tuned 0.8B version beats a fine-tuned 4B MinerU-Popo on that benchmark: 87.08 / 87.85 versus 84.18 / 85.64.

How it works

• Input assumption. Headings are already detected, put in reading order, and transcribed upstream. The method assigns levels only.
• Heading representation. Each heading's bounding box is drawn directly onto its page image. Coordinates are not passed as text.
• Memory. There is one anchor per level in the tree built so far. The anchor is the most recent heading predicted at that level. The pages holding those anchors form the memory, deduplicated, so the page count is at most the current tree depth. Every already-processed heading on those pages is also supplied, with its text, box and predicted level.
• Start. The first heading gets level 1. Later headings can also be level 1. The authors contrast this with relation-prediction methods such as D-O-C, which treat the first heading as the root.
• Constrained decoding. A heading's level must lie in {1, …, previous level + 1}. The tree then follows deterministically: each heading attaches to the most recent heading one level up.
• Granularity. The memory is fixed within a step, so the VLM can label all headings on one or more consecutive target pages in a single call, with the same +1 constraint applied in sequence. Grouping means fewer calls and gives look-ahead context.
• Anchor policy. The anchor-selection policy can be swapped out. "Most recent per level" is the only one reported.

Evidence

The metrics are Macro- and Micro-STEDS, given as the mean ± standard deviation of three runs (PaddleX was run once).

• Where the gain is. The largest gain is Micro on the long benchmark: +7.5 over Dynamic Chunking, +12 over Pseudo-ToC. On MPDocBench-Parse the margin over Pseudo-ToC is within roughly one standard deviation.
• Fine-tuned comparison. Each model was fine-tuned separately.
• The 0.8B model wins on CLDoc-HHRBench and on Overall Micro (85.32 vs 84.06).
• It loses Overall Macro (82.91 vs 83.95), MPDocBench-Parse (67.41/69.15 vs 71.87/71.13) and DocHieNet Macro.
• The authors present the 0.8B result as a feasibility demonstration.

Benchmark (CLDoc-HHRBench):

• 250 documents, 11,212 pages and 26,231 headings, across 10 document types.
• Means: 44.85 pages, 104.92 headings and depth 4.15. Maxima: 387 pages, 1,772 headings and depth 10.
• Existing test sets average about 7–14 pages and depth about 2–3.4.
• Sources are public PDFs and Word files, with no document-level overlap with the other benchmarks.
• Each document was labelled independently by two annotators and adjudicated by experts: 6 annotators and 3 experts, about 800 person-hours.

Limits

• Not end-to-end. Heading detection, reading order and OCR must come from elsewhere. Upstream errors are outside what is evaluated.
• Backbone unknown. The fetched text does not name the VLM used for the "Different Strategies" rows. It also does not say whether the Dynamic Chunking and Pseudo-ToC rows reimplement the MinerU-Popo and LingDT-VL-OCR approaches on the same backbone. Check this before comparing.
• Error propagation (my inference, not the authors'). The memory stores predicted levels, so an early mistake becomes context for later steps. The excerpt does not analyse this.
• Missing from the excerpt:
• Code, weights and licence.
• Whether CLDoc-HHRBench is released.
• Inference cost or latency.
• Fine-tuning data.
• An ablation of anchor strategies or prediction granularity. The text is truncated after Table 2.