Scaling Parameter-Efficient Fine-Tuning for Brain Encoding

What it claims

BrainLoRA adapts Qwen3-Omni with rank-16 LoRA (8.91M adapter parameters) and a lightweight readout. The authors report the best published score on the Algonauts 2025 challenge with about 5% as many trainable parameters as the prior state of the art, Gokce et al. (2026a), on the same backbone. The page reports results on both the in-distribution and out-of-distribution test sets. LoRA beats the frozen backbone on all nine multimodal backbones tested, which span 4.7B to 104B parameters. The authors say no backbone above 15B parameters had previously been adapted, rather than frozen, for brain encoding.

How it works

• Single-pass extraction. The prior method re-encoded an overlapping sliding window for every fMRI timestep, about 2,800 to 3,500 forward passes per clip. BrainLoRA encodes each 149 s window (100 TRs) in one causally masked pass and reads features at every timestep. This cuts feature extraction from about 700 to 2 GPU-hours on A800, roughly 350×. A 60 s context lead recovers most of the small accuracy drop caused by missing context at window start.
• Adapter. Rank-16 LoRA sits in the attention modules up to layer 32, and the decoder is truncated there. Adapters and readout train jointly for one epoch. IA3 and adapter layers give similar effects.
• Readout. Per-modality features are pooled at 2 Hz into eight log-spaced temporal-lag blocks. A linear projection then maps them to 1,000 Schaefer parcels, using 52.3M parameters. An alternative per-TR cross-attention readout over about 3.2k tokens adds 0.2M parameters. Ablations credit its gain mainly to access to the richer token set.
• Closed-form re-fit. After SGD training, the linear readout is re-solved with parcel-wise ridge regression from accumulated Gram matrices. The gain comes mostly from the re-estimation, not from the per-parcel regularization.
• Multi-LoRA kernel. A batched kernel trains many LoRAs jointly on one shared backbone. The 200+ model ensemble was trained this way.

Evidence

• Data. CNeuroMod/Algonauts 2025, with whole-brain fMRI from four subjects watching Friends. Season 6 is validation; Season 7 (ID) and held-out movies (OOD) are scored on the challenge server.
• Metric. Subject-mean Pearson r. Over 15 seeds, differences under 0.00077 cannot be separated. Single-model comparisons use a paired bootstrap over parcels.
• Gains over frozen. LoRA improves linear-readout r by 0.032 to 0.066 per backbone. Cross-attention adds about 0.003 more on average. Backbone size correlates with performance at r = +0.71 for frozen models and +0.60 for adapted ones.
• Single-pass versus sliding window. With LoRA, the single-pass setup surpasses the original sliding-window baseline.
• Ensemble. The 200+ model ensemble ranks first on the public post-challenge leaderboard as of October 2026, above unpublished entries too. Ensemble performance depends on size, individual model quality and diversity.

How to use it

Code is at https://github.com/zfying/brain-lora. The fetched text gives no install commands, weights or licence.

Limits

• The fetched text cuts off before Table 1, so absolute ID and OOD scores and the margin over the prior state of the art are not shown here.
• All results come from one benchmark (fMRI, audiovisual movies). The framework is claimed to apply to EEG and other modalities, but that is not tested.
• The modality encoders are locally bidirectional over 8 s of audio and 1 s of video. The authors measured small prefix differences and attribute them to bf16 MoE numerics rather than leakage of future stimuli.
• Frozen-readout single-pass features lose some accuracy without the context lead.