MEG-XL：基于长上下文预训练实现数据高效型脑信号到文本解码

MEG-XL: Data-Efficient Brain-to-Text via Long-Context Pre-Training

February 2, 2026

作者: Dulhan Jayalath, Oiwi Parker Jones

cs.AI

摘要

临床脑文本接口专为无法提供大量训练记录的瘫痪患者设计。预训练通过跨被试学习统计先验来提升数据效率化泛化能力，但这些先验高度依赖上下文环境。虽然自然语音可能持续数分钟，但现有方法大多仅用几秒上下文进行预训练。为此，我们提出MEG-XL模型，其每个样本使用2.5分钟脑磁图上下文进行预训练——比现有研究长5-300倍，相当于19.1万个标记，能捕捉更完整的神经上下文。在从脑数据解码词语的微调任务中，MEG-XL仅用少量数据（如1小时对比50小时）即可达到监督学习性能，并超越脑基础模型。我们发现长上下文预训练模型能学习到更适用于词语解码的表征。结果表明，长上下文预训练有助于利用其他方法不必要丢弃的扩展神经上下文。代码、模型权重及使用指南详见https://github.com/neural-processing-lab/MEG-XL。

English

Clinical brain-to-text interfaces are designed for paralysed patients who cannot provide extensive training recordings. Pre-training improves data-efficient generalisation by learning statistical priors across subjects, but these priors critically depend on context. While natural speech might unfold gradually over minutes, most methods pre-train with only a few seconds of context. Thus, we propose MEG-XL, a model pre-trained with 2.5 minutes of MEG context per sample, 5-300x longer than prior work, and equivalent to 191k tokens, capturing extended neural context. Fine-tuning on the task of word decoding from brain data, MEG-XL matches supervised performance with a fraction of the data (e.g. 1hr vs 50hrs) and outperforms brain foundation models. We find that models pre-trained with longer contexts learn representations that transfer better to word decoding. Our results indicate that long-context pre-training helps exploit extended neural context that other methods unnecessarily discard. Code, model weights, and instructions are available at https://github.com/neural-processing-lab/MEG-XL .