Source-preserving alignment for robust evidence localization in scientific PDFS

What it claims

The method highlights an extracted evidence quote on the original PDF page. It is deterministic and rule-based, with no learned parameters. Text is normalized only for comparison. A per-character provenance map links each normalized character back to its source text item and offset, so the highlight is clipped to the exact matched characters rather than to whole PDF.js text items. The authors also report human-verified accuracy: 91.9% of reviewed quotes were highlighted fully correctly, against 43.6% for PyMuPDF search and 19.1% for a precomputed bounding-box index.

How it works

flowchart TD
  Q[Evidence quote] --> N[Normalize quote + page stream<br/>keep source item, offset per char]
  N --> E{Exact match in<br/>normalized page?}
  E -- yes --> S[Project to source char spans]
  E -- no --> T[Tokenize, merge line-end hyphens]
  T --> L{Quote ≥5 tokens and<br/>best LCS score ≥ 0.8?}
  L -- yes --> S2[Keep only LCS-aligned tokens]
  L -- no --> X[Not found]
  S2 --> S
  S --> R[Clip rects by char range,<br/>merge same-line fragments]

• Normalization: Unicode compatibility normalization, dash/minus unification, repair of malformed encoding, mapping of scientific symbols, and removal of layout-only separators. The page text itself is never rewritten.
• Fallback: applies only to quotes of m ≥ 5 tokens. It tries candidate window lengths of round(α·m) for α ∈ {0.70, 0.85, 1.00, 1.15, 1.30}. Each window is scored as LCS(q,w) / max(m, ℓ) in tokens, and a match is accepted only if the score is ≥ 0.8. Hyphenation is merged only when the fragments form a word, so formula-like strings such as "ZSM-5" are kept intact. Unaligned tokens, such as citation markers, are not highlighted.
• Geometry: PDF.js exposes no per-glyph boxes, so the x-extent within an item is estimated proportionally from the item width (x₀ + w·c/|sᵢ|). The result is transformed by the item matrix and clipped to the viewport. Rectangles are merged only when they are nearby fragments on the same line, which avoids merging across columns or headers.

Evidence

• Corpus: 1,020 public chemistry papers paired with Markdown evidence files, giving 18,580 quotes (18.2 per paper). The pages include multi-column layouts, formulas and noisy text layers.
• Results (Table 1):
• Proposed: localization rate 92.1%, Correct 91.9%, Not found 7.9%.
• Text search: 43.6% / 43.6% / 56.4%.
• BBOX: 19.1% / 19.1% / 80.9%.
• Component ablation: normalization gives the larger gain, and the LCS fallback adds further recovery. The exact ablation numbers appear only in a figure, not in the text.
• Rendering ablation: this was run on 17,213 localized quotes with the matched spans held fixed. Highlighting whole items instead of clipped character ranges adds 62.8 characters per quote on average, which is 26.8% of highlighted characters.
• By quote length: the method peaks at 81–160 characters and declines after that, but stays well ahead of both baselines on long quotes. BBOX slightly beats it on the shortest quotes.

How to use it

The method assumes a PDF.js-style text layer that gives item strings and transform matrices. The authors say they provide the pairing rule, quote parser, BBOX records and comparison scripts, but the page gives no repository link or licence.

Limits

• Headline number: the abstract says 92.6% localization, but Table 1 says 92.1%.
• What the automatic metric measures: it counts any returned region, correct or not. The human labels are the meaningful figure.
• Human review setup: six evaluators from the authors' own lab, with one evaluator per output and no agreement figure. Figure 4's caption says 18,580 reviewed quotes, while the text says 18,572.
• Domain: tested only on chemistry papers.
• Geometry estimates: proportional width estimation gives loose boundaries with variable-width fonts.
• Reading order: complex layouts can break extraction order, and severe extraction errors exceed the normalization rules and the 0.8 threshold.
• Text layer required: a usable text layer must exist, so scanned PDFs without one are not handled.
• Short quotes: quotes under 5 tokens get no fallback.