Chunk Recomputation under Real Document Revisions: Measuring Update Cost and Retrieval Quality of Fixed, Recursive, and Content-Defined Chunking

What it claims

On 120 adjacent Wikipedia revision pairs (62 Chinese, 58 English), a small edit can force re-embedding of far more text than it touches. In one English article, an atom-level edit fraction of 0.13% made fixed-size windows re-embed 49% of the input, while paragraph-first chunking re-embedded 8%. At frozen parameters, non-overlapping fixed-size chunking costs 13 to 20 points more recomputation than recursive or content-defined chunking (CDC). Evidence recall stays within about two points. The authors attribute much of the recomputation of unchanged text to boundary drift. They support this with controlled edits and a verbatim-presence diagnostic.

How it works

An embedding counts as reused only if its complete input is byte-identical to an input produced for the previous version of the same document. Cross-document matches are not counted. The recompute fraction R_c is the token count of distinct new-version inputs that miss the cache, divided by the token count of all distinct new-version inputs. The fraction counts model input, not wall-clock time or price.

A no-edit control, which chunks the old version twice, must give R_c = 0. The pipeline also passes 20 hand-computed contracts covering inserts, deletes, moves, repeated text and empty documents.

All three chunkers are stateless and see only the current version:

• Fixed-size: greedy, non-overlapping windows of 304 tokens.
• Recursive: a 400-token target, split at blank lines, then line breaks, then sentence terminators followed by whitespace, with fixed windows as the fallback.
• CDC: the pinned Steadlith implementation (word-level Rabin fingerprints), with a 176/352 minimum/maximum and a 512-token cap.

Parameters were calibrated on six held-out articles so that median chunk lengths fell within 15% of each other, then frozen. Retrieval uses frozen BGE-M3 (CLS pooling, exact dot product) over a fixed multi-document pool. Whole chunks are packed into a 2048-token budget from at most 100 candidates. Evidence recall is binary per label.

Evidence

Paired differences are bootstrapped over articles, with 95% intervals:

• Recursive minus fixed: -12.7 points [-19.7, -6.2] in Chinese and -19.6 [-26.2, -13.2] in English.
• CDC minus fixed: -14.4 [-21.5, -7.9] in Chinese and -14.7 [-20.5, -9.2] in English.
• CDC minus recursive, Chinese: -1.7 [-6.8, +3.7], an interval that includes zero.
• CDC minus recursive, English: +4.9 [+0.9, +9.4], so recursive is cheaper there.
• Recall: all recall differences are between -1.9 and +0.9 points.

Across a cost-only grid of length-matched settings (15 Chinese, 14 English), fixed-size is always the most expensive. However, CDC-minus-recursive ranges from -0.7 to +5.5 in Chinese and from -2.4 to +6.2 in English, so which content-aware chunker wins depends on the parameters.

The gaps are widest for small edits (5% or less). For small English edits, recomputation is 46% for fixed, 25% for recursive and 27% for CDC. For small Chinese edits it is 62%, 45% and 36%. For extreme edits (over 50%), all methods converge, to about 92% in Chinese.

CDC failed on 4 of 62 Chinese articles, while the baselines never failed. Under every imputation the authors tried, Chinese CDC-minus-fixed stays between -13.5 and -10.1, with all intervals below zero.

Limits

• Statistics: the authors call the data a development sample. They run no hypothesis tests and make no non-inferiority claim. The edit-size bands hold only 2 to 28 articles each and are descriptive.
• Labels: human annotators assigned all 480 labels, but no second annotator adjudicated them.
• Edit-size proxy: the proxy uses difflib atoms rather than an audited alignment, and it overstates moved text.
• Recursive chunker in Chinese: Chinese terminators rarely have trailing whitespace, so oversized Chinese blocks fall back to fixed windows.
• Chunk length: recursive chunks are the longest, or tied for longest, in 101 of 116 articles. The realised median length ratio is 1.17 rather than equal.
• Other corpora: PDF and arXiv-style corpora add extraction noise that a byte-identical rule treats as edits. The arXivEdits check is cost-only.
• Scope: stateful reuse methods such as KARA are not compared.
• Fetched text: the page text is truncated partway through the results.