MEG-XL：通过长上下文预训练实现数据高效型脑到文本解码

MEG-XL: Data-Efficient Brain-to-Text via Long-Context Pre-Training

February 2, 2026

作者: Dulhan Jayalath, Oiwi Parker Jones

cs.AI

摘要

臨床腦機文本介面專為無法提供大量訓練記錄的癱瘓患者設計。預訓練技術通過學習跨受試者的統計先驗來提升數據效率化的泛化能力，但這些先驗高度依賴上下文環境。雖然自然語言的表達可能持續數分鐘，但現有方法大多僅使用數秒上下文進行預訓練。為此，我們提出MEG-XL模型，該模型對每個樣本進行長達2.5分鐘的腦磁圖上下文預訓練，上下文長度達到既往研究的5-300倍，相當於19.1萬個標記，能捕捉更完整的神經上下文信息。在從腦數據解碼詞語的任務中進行微調後，MEG-XL僅需極少數據（如1小時對比50小時）即可達到監督式學習的性能水平，並超越各類腦科學基礎模型。我們發現，採用長上下文預訓練的模型能學習到更適用於詞語解碼的表徵。研究結果表明，長上下文預訓練有助於充分利用其他方法不必要捨棄的延伸神經上下文信息。相關代碼、模型權重及使用指南已開源於https://github.com/neural-processing-lab/MEG-XL。

English

Clinical brain-to-text interfaces are designed for paralysed patients who cannot provide extensive training recordings. Pre-training improves data-efficient generalisation by learning statistical priors across subjects, but these priors critically depend on context. While natural speech might unfold gradually over minutes, most methods pre-train with only a few seconds of context. Thus, we propose MEG-XL, a model pre-trained with 2.5 minutes of MEG context per sample, 5-300x longer than prior work, and equivalent to 191k tokens, capturing extended neural context. Fine-tuning on the task of word decoding from brain data, MEG-XL matches supervised performance with a fraction of the data (e.g. 1hr vs 50hrs) and outperforms brain foundation models. We find that models pre-trained with longer contexts learn representations that transfer better to word decoding. Our results indicate that long-context pre-training helps exploit extended neural context that other methods unnecessarily discard. Code, model weights, and instructions are available at https://github.com/neural-processing-lab/MEG-XL .