Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text

What it claims

Most of the gain from jointly decoding all words in a sentence (d'Ascoli et al., 2025) comes from word timing. It does not come from brain activity. The joint decoder's 3-second windows start at each word onset, so neighbouring windows overlap. The offset between shared samples gives away the interval between word onsets, and word duration narrows down the candidate words. The fix is to decode each window independently. With that change, two standard techniques start to pay off: aggregating repeated observations of a word and adding an LLM linguistic prior. The resulting recipe, SimpleB2T, reaches 65.6% WER with one observation per word and 36.6% with five, on a "core subset of clinically motivated sentences".

How it works

• Baseline (d'Ascoli et al.): cut a 3 s M/EEG window at each word onset. A small CNN with temporal pooling encodes each window, then a 16-layer, 16-head bidirectional transformer processes the whole sentence (~200M parameters). Outputs are trained contrastively against frozen T5-large word embeddings. At test time, the predicted word is the nearest neighbour by cosine similarity over a candidate vocabulary.
• Why it leaks: adjacent windows satisfy x_i[n+d] = x_{i+1}[n]. The onset gap d can therefore be recovered just by minimising a sum of squared differences. On their data:
• 99.9996% of adjacent word pairs overlap, sharing 90.6% of samples on average.
• The onset interval correlates with word duration at r = 0.90.
• SimpleB2T encoder: keeps the same windows, T5 targets and contrastive loss. Each window goes alone through a small CNN with temporal pooling and a residual MLP (~20M parameters). The model has no access to the other windows. The fetched text is cut off before the aggregation and LLM-prior details, so those are not described here.

Evidence

The data is LibriBrain100, subject 0: about 80 h of MEG recorded while listening to audiobooks, podcasts and sentences. Splits share no stimuli and are separated by session. The metric is balanced top-1 accuracy over the 50 most frequent words.

Further results:

• The effect reproduces across 32 more subjects and on additional datasets (per the appendices).
• Retraining on non-overlapping sentence constructions gives no advantage over isolated decoding.
• The joint decoder is much more accurate on words with consistent durations (r = −0.90 for words with duration σ < 50 ms).
• Its top-10 candidates match the specific test occurrence's duration: duration error drops by ~13 ms, against ~0.7 ms for the isolated decoder.

How to use it

• Code and notebooks: github.com/neural-processing-lab/SimpleB2T
• Benchmark and data tooling: github.com/neural-processing-lab/pnpl

The fetched text gives no install commands, licence or compute figures.

Limits

• Your own results: if you built on the d'Ascoli joint-decoding setup (the paper lists about a dozen follow-ups and benchmarks that did), reported gains may be inflated by the same shortcut. Rerun with isolated or synthetic-signal controls.
• Masking doesn't help: truncating or masking windows at word boundaries does not fix the leak, because it reveals the same timing information.
• Joint decoding isn't ruled out: it may still help when words are far enough apart that windows don't overlap.
• Known onsets only: the setting assumes word onsets are known. Onset-free decoding is a harder task that is not evaluated.
• Invasive comparison: "approaching invasive performance" is qualified by the authors themselves as being under different conditions.
• Narrow headline: the WER figures come from a core sentence subset, and the five-observation figure needs repeated presentations of each word.
• Truncated page: the fetched text stops partway through Section 3. Aggregation, the LLM-prior method and the full WER tables were not available for this note.